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D eription 

FIELD OF THE INVENTION 

s The present invention relates generally to diagnostic testing, and more particularly to methods and apparatus for 

diagnostic testing using evidential reasoning and for determining when sufficient information has been collected to 
make a valid diagnosis. 

BACKGROUND OF THE INVENTION 

10 

The assignee of the present invention has previously developed a portable interactive diagnostic tester for fault 
isolation in complex electromechanical systems called POINTER. POINTER utilizes a model-based, matrix-implement- 
ed inferential reasoning method of diagnostic analysis. The diagnostic model is created using the STAMP (System 
T stability and Maintenance Program) program also available from the assignee of the present invention, ARINC Re- 

is search Corporation. STAMP is an interactive modeling tool designed to permit analysis of the testability of a system 
and .generation of static fault isolation strategies. STAMP assumes that the system design is known, tests can be 
specified that measure the goodness and badness of system elements, and sequences of test outcomes can be used 
to localize or isolate failed system elements. The STAMP diagnostic model describes the flow of information through 
a system to be diagnosed. The information flow is represented as a set of diagnostic tests (information sources), a set 

20 of diagnoses (conclusions to be drawn), and a set of logical relationships between the tests and the conclusions. In 
addition, the model includes parameters for weighting a test selection optimization process, grouping tests and con- 
clusions, and specifying types of tests to be performed (e.g. functional tests, conditional tests, or asymmetric tests). 
Under the STAMP methodology, an assumption is made that only one fault exists or that specific multiple faults can 
be specified and treated as a single fault, and thus that only one conclusion is true for any one diagnostic analysis. (If 

25 th system under test actually has multiple faults, they can usually be successively isolated by repeating the diagnostic 
analysis, unless a "hidden" failure causes the failure which is isolated or a multiple failure looks like an independent 
single failure.) Evidence from an individual test flows to a set or sets of diagnostic conclusions as either support or 
denial evidence in accordance with the defined logical relationships. That is, a logical proposition consisting of a set 
of dependency, or support, functions is established or defined which identifies the conclusions related to a given test 

30 and allocates the evidence generated by a test measurement to the set of related conclusions. An example of a higher- 
order proposition relating a test t s to a set of conclusions c v c 2 , ....c„ is: 
If (t 5 ) then c, or c 2 or ....c n implication 
If (~Cf and ~c 2 and ~.. .c n ) then ~t 5 negation 
If (~t 5 ) then ~c, and ~c 2 and ....""c n symmetry 

35 Each unique test has two different support functions, one for each type of evidence (support or denial). There is 

thus a set of conclusions that are supported and a set of conclusions that are denied by a given test outcome. That is, 
every conclusion in the model is supported or denied by each test outcome. A test supports a conclusion when the 
test outcome is consistent with a conclusion value of true. A test denies a conclusion when its outcome is not consistent 
with a conclusion value of true. In addition, component-to-component dependencies in the system are not modelled, 

40 and thus an assumption is made that the evidence for a given conclusion is determined solely by the tests that are 
related to it, not by other conclusions that may resemble its component parts. 

The test measurement results are evaluated and specified as indicating that the tested system element is "Good, 
" "Bad" or "Unknown" (results not available). A Bad test outcome is equated to be a logically True value of the proposition 
that corresponds to the test; and a Good test outcome equates to a logically False value of the corresponding test 

45 proposition. In the example given above, testing would evaluate the outcome of and then the truth or falsity of other 
propositions and conclusions are inferred using the applicable set of support functions. Thus, the STAMP process can 
be generally described as solving a set membership problem. If a given test is Bad, it implies one of n higher-order 
causes. If another test is Good, it implies that all of the potential causes of its being Bad are not possible conclusions, 
so that the complementary set (all potential causes that are still possible) is possible. STAMP operates to ascertain 

50 the intersections of the possible conclusion sets. 

The STAMP paradigm has proven successful in solving a large number of testability and fault-isolation problems. 
However, many important problem domains involve uncertainty in both the statements of relation between tests and 
conclusions and the interpretation of the test measurements. For example, the model assumes that the relationships 
between th information sources and the diagnostic conclusions are clearly discernible, well defined, and unambiguous. 

ss it may not be possible, though, to establish a model which is both accurate and complete, i.e., a mod I in which each 
conclusion is properly relat d to each t st result, and all of the conclusions which should b related to a test result are 
specified. Further, the test information may be ambiguous, difficult to interpret, or the user doing the testing and inter- 
preting may have inadequate skill levels to produce reliable test results. Thus, an indicator is ne ded which measur s 
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how well the modelled relations are fitting the problem to which they are being applied. A number of differ nt approaches 
have been developed in the artificial intelligence field to deal with the unc rtainty created by these problems. Generally 
referred to as "reasoning under uncertainty," exampl s of such approaches include th use of c rtainty factors, prob- 
abilistic reasoning and weighted causal n tworks. In addition, there are various logics that take into account aspects 
s of uncertainty such as predicate calculus, multi-valued logics, modal logics, non-monotonic reasoning, intuitional logic, 
truth maintenance, "fuzzy" logic, and Dempster-Shafer evidential reasoning. 

However, it is difficult in general to integrate reasoning under unc rtainty concepts with the massive structure and 
processes associated with a complex, matriximplemented inferential reasoning process like STAMP. Further, although 
the Dempster-Shafer evidential reasoning approach is well adapted for use in an inferential diagnostic process like 
10 STAMP, even this approach to dealing with uncertainty creates a number of serious problems. The Dempster-Shafer 
technique is flawed in the way it handles total uncertainty. If any test is performed that provides any evidence in support 
of some conclusion, then uncertainty is reduced-even in the event of a conflict with known information. Ultimately, as 
further testing is conducted, uncertainty disappears altogether, independent of the test results. Also, applying uncer- 
tainty calculations to an inferential system makes the criteria for determining when a diagnostic conclusion may be 
is properly drawn unclear. There is thus a need to determine at what point enough information has been gathered to 
declare a valid diagnosis, or, in the context of fault isolation, to determine the point at which additional testing is providing 
minimal useful new information. 

Examples of commonly used diagnostic approaches include protocol or decision tree based systems and simula- 
tion models; and so called "intelligent" systems such as rule-based expert systems as discussed below. 

20 

Shortllffe, E.H. Computer Based Medlcial Consultations: MYCIN. American Elsevier, New York, 1976. 

The first "successful" artificial intelligence application was the MYCIN system developed by Edward Shortliffe at 
Stanford University. This book was based on Shortliff e's Ph.D. Thesis and describes the use of heuristic rules, backward 
25 chaining, and an uncertainty mechanism called "certainty factors" as applied to the problem of diagnosing infectious 
diseases. The MYCIN system was specifically designed to serve as a decision aid for physicians and physician as- 
sistants in the diagnostic process. This system was completely passive and gave no indications of whether more testing 
was required to make a diagnosis. Rather, only confidence levels were provided with recommended diagnoses and 
treatments. 

30 

Davis, R., "Diagnostic Reasoning Based on Structure," Artificial Intelligence, Vol. 32, pp. 97-130, 1987. 

In an attempt to move from the medical domain to the electronics domain, Randall Davis developed a model-based 
approach to diagnosis focusing on structural and behavioral characteristics of the system being tested. The models 
35 used structure to define signal flow through the circuit and behaviour to reason about responses to tests. Davis' model- 
based approach was also passive and included no mechanism for reasoning under uncertainty. Later work on the 
model applied entropy-based search to optimize testing and moved the approach from passive to active, but to our 
knowledge, uncertainty was still left out, Consequently, termination wasdeterministic, meaning it was based on applying 
test results to sets of hypotheses until the smallest set that could explain the test results was obtained. 

40 

Pipitone, F., "The FIS Electronics Troubleshooting system," IEE Computer, pp. 68-75, 1986. 

While working at the Naval Research Laboratory (NRL), Frank Pipitone developed a model-based diagnostic sys- 
tem (the Fault Isolation System -FIS) which was based strictly on structural characteristics of electronic systems. These 
45 models are called "dependency models" in the diagnostics literature. The approach was heavily influenced by Davis, 
but did not include the behavioral knowledge. Further, Pipitone added probabilistic reasoning to the inference process. 
FIS was an active diagnostic system (i.e. test choice was optimized), but termination was based on testing with certain 
outcomes as in Davis' approach. In other words, the uncertainty of the answer had no bearing on whether or not FIS 
would provide an answer, thus degrading the accuracy in the answer. 

so 

Allred, Lloyd, "Neural Networks in Automatic Testing of Diode Protection circuits," AUTOTESTCON '89, pp. 
181-186. 

Neural networks are being applied in increasing frequency to the problem of electronic system diagnosis. This 
55 paper by Lloyd Allred provides an example of one application of neural networks for circuit diagnosis as applied to 
Zener diodes. The described approach involves extracting relevant features from the performance of the diode and 
using these features to train a neural network. The neural network was based on the architecture of a self-organizing 
feature map in which feature values are grouped into cells bas d on distance. The cells were th n labelled with fault 
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identifiers so that when a c II fired, the corresponding fault was identified. Cell sizes were adjust d through training so 
that each cell only identified a single fault. The approach described yields a network that is highly specific, system 
sp cific, and test specific. To be used for testing other units, a new network would ne d to be constructed. This, in fact, 
is typical of neural network applications for diagnosis, 
s Examples of known methods which can be used to determine levels of certainty (or "pseudo" probabilities) hav 

be n cited hereinabove, and includ the following: 

McLeish, M., "A Note on Probabilistic Logic," AAAI-88, pp. 215-219, 1988 

io The question of how one applies probabilistic logic to reasoning systems is addressed in this paper. The question 

addressed is not specific to diagnostics but concerns whether two forms of probabilistic reasoning (namely maximum 
ntropy and projection) yield the same result. The benefit of the article is on analysis of reasoning performance to 
determine validity of the underlying mathematics. It addresses neither efficiency of reasoning nor termination criteria. 

is Zadeh, L., "Fuzzy logic," IEEE Computer, pp. 83-93, April 1988 

Lotfi Zadeh defined a new method of reasoning under uncertainty that he called "Fuzzy Logic." Fuzzy logic, as 
defined in this paper, is based on the notion of a fuzzy set. Rather than declaring an element to be a member of some 
set, the mathematics allows a "level" of membership to be associated with the element. Fuzzy logic then consists of 
20 methods for combining fuzzy membership values to determine, given multiple pieces of evidence, whether or not some 
set is characterized by the data. 

Shafer, G., A Mathematical Theory of Evidence, Princeton University Press, pp. 35-73, 1976 

25 Believing Bayesian probability theory was powerful but too unwieldy to apply in practice, Shafer devised a modi- 

fication to Bayesian inference in which he analyzed available evidence to determine levels of belief and denial in a 
hypothesis. Shafer"s theory of evidence provided a means for determining belief intervals for the hypotheses rather 
than specifying a single probability value. Shafer believed single probability values were too difficult to interpret because 
they did not provide sufficient information about the reasoning process. This work was not explicitly applied to diagnosis; 

30 however, its widest application was in traditional classification tasks (of which diagnosis is an example). The mathe- 
matics did not include methods for optimization or termination but considered all available evidence. 

Dempster, A.P., "A Generalization of Bayesian Inference," Journal of the Royal Statistical Society, Series 8, 
pp. 205-247, 1968 

35 

Dempster extended the work by Shafer to provide a method for combining evidence from multiple sources. In 
particular, Dempster defined a "rule of combinations" that permitted an iterative analysis of evidence. The resulting 
reasoning mechanism is now referred to as "Dempster-Shafer theory" and forms the basis of many reasoning systems 
most notably sensor fusion systems. Again, Dempster assumed the availability of the needed evidence and did not 
40 focus on optimization for evidence selection or termination of the reasoning process. 

The rule-based expert system is perhaps the most common of the "intelligent" diagnostic systems. Experts are 
interviewed to determine how they diagnose, and rules are written to describe the process. Uncertainty is most fre- 
quently handled using probabilistic inference or certainty factors. 

The objects of the invention are achieved by the apparatus as set forth in claim 1 . The dependent claims are 
45 directed to preferred embodiments of the invention. 

In accordance with the present invention there is provided automatic test data evaluation apparatus for use in a 
diagnostic tester that declares a diagnosis on the basis of localisation or isolation of failed system elements of a system 
having a defined number of functional elements, said apparatus comprising: 

so means for selecting the next test to be performed on the system to be tested; 

means for generating the corresponding test data and supplying them to the system to be tested; 
means for sensing and collecting predetermined system operating parameters; 

evaluating means, responsive to at least one inputted test signal corresponding to test data St relating to at least 
one predetermin d parameter of the system, for producing using a probabilistic evidential reasoning m thod: 
55 first and second candidate signals corresponding respectively to first and second possible diagnoses of the con- 

dition of th system respectively having the first and second highest levels of certainty of being valid, and 
first and second certainty signals corresponding respectively to values of said first and second highest levels of 
certainty; and 
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characterized by testing sufficiency means, responsiv to said first and second c rtainty signals, for producing 
an output signal indicative of whether sufficient test data has been evaluated to declare a diagnosis, said testing suf- 
ficiency means comprising a n ural network having an output nod and at least thr input nodes resp ctiv lyr c iving 
said first and second certainty signals and a predetermined bias signal as an input, said input nodes having weight d 
5 outputs, the output weights corresponding to a model of expert data correlating combinations of certainty signals with 
at least one expert opinion regarding th ability to declare a diagnosis on the basis of ach combination of c rtainty 
signal values. 

The present invention provides improved diagnostic testing methods and apparatus which deal with uncertainty 
and overcome the deficiencies of conventional approaches. 
10 The present invention also provides improved evidential reasoning based diagnostic testing methods and appa- 

ratus which overcome the limitations of the Dempster-Shafer approach. 

The present invention also provides methods and apparatus for determining the sufficiency of testing in diagnostic 
testing methods and apparatus which deal with uncertainty. 

The present invention also provides a neural network method and apparatus for determining the sufficiency of 
is testing in diagnostic testing methods and apparatus dealing with uncertainty wherein the neural network can be trained 
to haye generic application to different diagnostic models and different systems. 

Typically said evaluating means also produces an uncertainty signal corresponding to a measure of the uncertainty 
that the evaluated at least one test signal can be validly evaluated; and 
said neural network comprises: 

20 

a set of first, second, third and fourth input nodes each having first, second and third outputs weighted by a pre- 
determined fixed weight, said first and second input nodes respectively receiving said first and second certainty 
signals as an input, said third input node receiving said uncertainty signal, and said fourth input node receiving 
said predetermined bias signal; 

25 a set of first, second, third and fourth intermediate nodes each having an output weighted by a predetermined fixed 

weight, and said first intermediate node receiving the sum of the first weighted outputs of said input nodes as an 
input, said second intermediate node receiving the same of the second weighted outputs of said input nodes as 
an input, said third intermediate node receiving the same of the third weighted outputs of said input nodes as an 
input, and said fourth intermediate node receiving said predetermined bias signal; and 

30 an output node, responsive to the sum of said weighted outputs of said intermediate nodes, for producing said 

output signal, the output weights of said input and intermediate nodes corresponding to a model of expert data 
correlating combinations of certainty and uncertainty signal values with at least one expert opinion regarding the 
ability to declare a diagnosis on the basis of each combination of certainty and uncertainty signal values, said 
testing sufficiency means thereby also evaluating said uncertainty signal to produce said output signal. 

35 

Typically said evaluating means also produces a signal S N corresponding to a measure of the level of certainty 
that a diagnosis not corresponding to said first or second candidate signals or to said uncertainty signal is a valid 
diagnosis; and 

said testing sufficiency means also evaluates said signal S N to produce said output signal. 

40 The diagnostic tester typically further comprises means for storing a model of predetermined information flow 

relationships within the system under test; means for associating the test signals with sets of possible causes in ac- 
cordance with the model, and for ascertaining the intersections of the sets of possible causes for combinations of test 
signals, the candidate signals being derived from the intersections; and means for determining for each candidate 
signal a measure of the level of certainty that it represents a valid diagnosis. 

45 Typically the diagnostic tester further comprises means for defining a plurality of conclusions corresponding to 

possible diagnoses of the system; means for assigning a confidence value to each successively inputted test signal; 
means for calculating the support and denial for each conclusion based on the confidence value of the currently inputted 
test signal; means for determining if the currently inputted test signal conflicts with a then selected set of said conclu- 
sions corresponding to candidate signals; means for obtaining a measure of internal conflict based on accumulated 

so total support values and the current test signal support values for all of the conclusions; means for computing the 
present value of the uncertainty signal based on the number of test signals evaluated, if the present test signal is not 
a conflict; and based on the number of test signals evaluated and the number of conflicts, if the present test signal is 
a conflict; means for computing a new total support value for each conclusion based on the old support value, the 
present test signal support value and confidence value, an old uncertainty value, and the measure of int rnal conflict; 

55 means for computing a new uncertainty value based on the old uncertainty valu , the present test signal confidenc 
value and the measure of internal conflict; means for normalizing the total support value for each conclusion based on 
the old total support value, and the present uncertainty signal value; means for computing a new total denial for ach 
conclusion based on an old total denial and the present t st signal denial value; m ans for computing th present 
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plausibility for each conclusion based on the present total denial and the number of t st signals evaluated; means for 
selecting the current set of candidate signals based on the present total support and the plausibility for each conclusion. 

While a pur ly inf r ntial form of STAMP provides a very good method for optimizing the choice of the next test 
to maximize the efficiency of the diagnostic process, this method breaks down when an vidential r asoning approach 
s is adopt d. Heretofore, the problem of test selection has not b en addressed in the field of evidence combination. 
Second, the process of identifying the conclusions, observations and relationships that should be involved in a given 
evidential diagnostic process is much more difficult than with a strictly inferential process. 

Typically the diagnostic tester further comprises an arrangement for testing a system according to a set of defined 
relationships between a plurality of sources for providing information about the system and a plurality of conclusions 
w about the nature or condition of the system, said tester comprising first means for successively selecting an information 
source from among the plurality of information sources for interrogation according to the relative amount of information 
provided by each information source until all of the information sources have been selected or the information which 
would be provided thereby has been determined; 

second means for successively further selecting an information source from among the plurality of information 
75 sources for interrogation according to which information source will increase the level of certainty that a selected can- 
didate conclusion constitutes a valid diagnosis of the system being tested. 

These and other features and advantages of the present invention are disclosed in or apparent from the following 
detailed description of preferred embodiments. 

The preferred embodiments are described with reference to the accompanying drawings, wherein like features 
20 have been designated with like reference numbers, and wherein: 

Fig. 1 is a diagrammatic view of a first embodiment of a diagnostic tester according to the present invention. 
Fig. 2 is a schematic block diagram of the basic functional components of the diagnostic tester of Fig. 1. 
Fig. 3 is a schematic block diagram of the control and diagnostic analysts CPU of Fig. 2. 
Fig. 4 is a diagrammatic view of a second embodiment of diagnostic tester according to the present invention. 
25 Fig. 5 is a schematic block diagram of an automatic system tester incorporating diagnostic testing apparatus in 

accordance with the present invention. 

Fig. 6 is a schematic block diagram of a system controller incorporating diagnostic testing apparatus in accordance 
with the present invention. 

Fig. 7 is a schematic block diagram of the controller and diagnostic test signal evaluating sections of a diagnostic 
30 tester according to the present invention. 

Fig. 8 is a general flow chart of a diagnostic process performed by a diagnostic tester in accordance with the 
present invention. 

Fig. 9 is general flow chart of a preferred diagnostic process performed by a diagnostic tester in accordance with 
the present invention. 

35 Fig. 10 is a more detailed flow chart of a preferred test selection process performed by a diagnostic tester in 

accordance with the present invention. 

Fig. 11 is a more detailed flow chart of a preferred evidential reasoning based process performed by a diagnostic 
tester in accordance with the present invention for selecting a conclusion from among a plurality of conclusions as a 
diagnosis based on inputted test signals. 
40 Fig. 12 is a diagrammatic illustration of a first embodiment of a neural network in accordance with the present 

invention for determining the sufficiency of the testing by a diagnostic tester. 

Fig. 13 is a diagrammatic illustration of a second embodiment of a neural network in accordance with the present 
invention for determining the sufficiency of the testing by a diagnostic tester. 

45 DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Referring to Figs 1-6, the diagnostic testing methods and apparatus of the present invention advantageously are 
incorporated in diagnostic testers which are adapted to receive test data input from an operator or from other sources 
(Figs. 1-4); in automatic system testers which are adapted to automatically both test a system and analyze the test 

50 results (Fig. 5); or in automatic system controllers which sense the condition of the controlled system and control the 
operation of the system in response thereto (Fig. 6). The present invention may also readily be embedded within 
systems having self-diagnosis capabilities as part of the self diagnosis architecture. As shown in Figs. 1-3, one form 
of manual input diagnostic tester which is portable has a separate housing 11 which contains the tester components; 
a display 12 mounted in housing 11 for displaying t st results, diagnoses, user prompts and similar types of information; 

55 a keyboard ar a 13 comprising a plurality of data entry keys 14 and a plurality of command keys 15 for ent ring test 
data and user commands to control the op ration of the tester; conventional signal g n rating circuitry 16 responsive 
to actuation of keys 14 and 15 for generating digital electrical test and command signals, respectively, corresponding 
to the test data and commands enter d by the operator; and a control and diagnostic analysis central processing unit 
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(CPU) 17 for controlling the operation of the tester and performing the diagnostic analysis routin s stored therein. As 
is conventional, CPU 17 comprises a processor 17a for performing control and analysis operations under control of 
an op rating program stored in a read-only memory (ROM) 17b, and a further data m mory 17c for storing inputted 
test and command signal data and the data generated in th course of operation of processor 1 7a. 

s Referring to Fig. 4, another form of a diagnostic tester in accordance with th present invention comprises a con- 

ventional general purpose or dedicated computer system 20 comprising a display 22, a CPU 24 programmed to perform 
diagnostic analysis routines, and an input interface 26 for receiving test data and command signal input. As will be 
appreciated by those skilled in the art, CPU 24 has the same general functional organization as CPU 17 shown in Fig. 
3. Input interface 26 advantageously comprises a conventional keyboard or other user interface device for generating 

io test data and command signals in response to operator actuation of the device, or a conventional peripheral interface 
device for receiving test data and command signals from a peripheral source such as a storage memory or system 
controller; and converting, if appropriate, the test data and command signals to digital signals in a form compatible for 
processing by CPU 24. 

Referring to Fig. 5, an automatic system tester 30 for testing a system A which incorporates a diagnostic tester in 

is accordance with the present invention comprises a controller and diagnostic test signal analyzer 32 for generating 
control signals to control the operation of tester 30 and for analyzing test signals indicative of the nature or condition 
of system A which are generated by tester 30; a test generator system 34 for performing predetermined tests on system 
A responsive to control signals generated by controller/analyzer 32 and for generating the test signals corresponding 
to the test results; and a display 36 responsive to control signals from controller/analyzer 32 for displaying test infor- 

20 mation, diagnoses, user prompts and the like. 

Referring to Fig. 6, a diagnostic tester in accordance with the present invention is shown incorporated in a system 
controller 40 controlling the operation of a system A. System controller 40 comprises a sensor/control unit 42 for sensing 
predetermined system operating parameters and for generating control signals for controlling the operation of system 
A; and a diagnostic tester 44 for evaluating the sensor data and generating output signals used by sensor/control unit 

25 42 for generating the control signals. 

The diagnostic tester of the present invention has broad application in determining the nature or condition of a 
variety of systems, ranging, for example, from determining faults and malfunctions in electronic or electromechanical 
systems or manufacturing processes, to selecting the appropriate control response according to the sensed state of 
a controlled system, to identifying a disease or other abnormal condition in an organism, to identifying objects within 

30 a space, to classifying animals or other objects or materials according to a taxonomy or other classification scheme. 
For the sake of clarity, the diagnostic tester of the present invention will be specifically described hereinbelow in the 
context of fault diagnosis of complex systems, wherein malfunctioning systems having a plurality of functional compo- 
nents or elements are analyzed to isolate the source of a fault. 

Referring to Fig. 7, the basic preferred functional configuration of processor 17a, CPU 24, controller and diagnostic 

35 test signal analyzer 32, and the control portion of sensor/control unit 42 and diagnostic tester 44 comprises a controller 
section 52, a test signal evaluation section 54, and a sufficient testing evaluation section 56. Controller section 52 
generates control signals S Con for controlling the associated display 1 2, 22, 36 and/or test generator 34 and system A. 

Test signal evaluating section 54 evaluates inputted test signal data St resulting from testing or sensing of some 
aspect, referred to hereinafter as a parameter, of system A, and produces a plurality of n candidate signals S c corre- 
sponding to n possible diagnoses of, or diagnostic conclusions about the condition of system A, where n>2, and each 
candidate signal S c has an associated level of certainty of representing a valid diagnosis, that is, a diagnosis having 
associated therewith an acceptable level of confidence that the diagnosis is correct. Hereinafter, the candidate signals 
having the first and second highest levels of certainty of being valid are denoted S CP1 and S CP2 . Test signal evaluating 
section 54 also produces first and second certainty signals S P1 and S P2 corresponding respectively to measures or 

45 values of the first and second highest levels of certainty associated with candidate signals Sc P1 and S CP2 . Test signal 
evaluating section 54 also preferably produces uncertainty signals Sy corresponding to a measure of the uncertainty 
that the evaluated test signals S^- can be validly evaluated to produce a diagnosis. 

Sufficient testing evaluating section 56 produces an output signal S s indicative of whether sufficient test data has 
been evaluated to declare a diagnosis based on at least the certainty signals S P1 and S P2 , and preferably also based 

50 on uncertainty signal Su. In addition, test signal evaluating section 54 advantageously also generates an associated 
signal S N corresponding to a measure of the level of certainty that a conclusion other than S CP1 , S CP2 and Sy is the 
correct diagnosis, and the output signal S s of sufficient testing evaluating section 56 is also based on signal Sfj. 

Referring to Fig. 8, in general, the procedure followed to make a system diagnosis in accordance with the present 
invention comprises the steps of selecting information sources (the diagnostic tests/sensed param ters) to be valu- 

55 ated (step SS1 ); collecting information via these information sources (performing the selected test/sensing the selected 
param ter and evaluating the results to produce test data) (step SS2); analyzing the information (test data) coll cted 
(step SS3); drawing a candidate conclusion or conclusions about the situation examined (generating candidate signals 
S c ) (step SS4); and determining whether sufficient testing has been conducted to declare one or more of th conclu- 
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sions as the diagnosis, or to d clare that further t sting is unlikely to establish a diagnosis (Step SS5). If the step SS5 
determination is positive, then further testing is terminated and a diagnosis is declared, if appropriate (step SS6). If the 
step SS5 determination is negative, then the procedure is repeated by returning to step SS1, unless some other ter- 
mination crit rion is satisfied. 

s In accordance with the present invention, any conventional method can be used to select the tests to be performed/ 

parameters to be sensed, and to collect the test data. Similarly, any conv ntional method can be used to analyze the 
test data and draw candidate conclusions (generate candidate signals S c ) which also provide a measure of the levels 
of certainty that the various candidate conclusions are valid. 

A preferred model-based method of test selection and test data analysis for fault diagnosis of complex systems 

10 in accordance with the present invention which employs a modified form of Dempster-Shafter evidential reasoning will 
now be described. In accordance with this method, the problem of diagnosis is approached as a collection of subprob- 
lems: 

(1 ) Modeling : A diagnostic problem is treated as a modeling problem in which available information sources, the 
is desired set of diagnostic conclusions, and their interrelationships are specified. 

(2) Optimization : The information sources are selected for consideration in a way that optimizes the diagnostic 
process according to one or more criteria. 

(3) Logical Inference : Test data derived from the information sources is used to draw inferences about other infor- 
mation sources and about the diagnostic conclusions in the model. The inferences are drawn based upon the 

so relationships specified in the model. 

(4) Statistical Qualification : Statistics describing the support and plausibility of each diagnostic conclusion in the 
model are computed based upon the information sources examined, the information collected, and the confidences 
in the reliability of that information. 

(5) Statistical Inference: Statistics generated from collecting information from each information source are com- 
25 bined to determine the total support and plausibility for each diagnostic conclusion in the model. 

These statistics are then used to estimate the probabilities that the corresponding conclusions constitute correct 
diagnoses. The resulting probabilities are then examined in the manner described in more detail hereinafter to deter- 
mine if sufficient information has been collected to make a confident, valid diagnosis, and if not, whether further test 

30 data should be collected. If the determination is positive because sufficient information is deemed to have been col- 
lected, then further testing and evaluation are terminated and the diagnosis corresponding to candidate signal Sc Pl 
having the highest certainty level of being valid is declared as the diagnosis. Alternatively, if the determination is positive 
because the uncertainty due to conflict is such that no confident valid diagnosis can be made in the near term, then 
further testing and evaluation are terminated and the diagnosis which is declared is that corresponding to uncertainty 

35 signal Sj, i.e., that a valid diagnosis cannot be made. If the determination is negative because insufficient information 
has been collected, then further testing and evaluation are conducted until a positive determination is obtained, unless 
another termination criterion is satisfied. Two such criteria are whether a trend analysis indicates that additional test 
data collection is not improving the ability to select a conclusion as the fault diagnosis; and whether all information 
sources have been used a predetermined number of times without a determination being made that sufficient infor- 

40 mation has been collected. Preferably, the repetition limit varies for each test according to the confidence values (de- 
scribed in more detail hereinafter) assigned to the test. In the preferred embodiment, the standard limit is 4 unless the 
value of the greater of the Good and Bad base confidences is less than 0.85. Then the limit is reduced as follows: 
if 0.75 < Base < 0.85, then the limit = 3; 
if 0.6 < Base < 0.75, then the limit = 2; and 

45 jf Base < 0.6, then the limit = 1 ; 

where Base is the greater base confidence value. 
A typical fault isolation determination using a diagnostic model generated by the aforementioned STAMP program 
involves choosing tests, performing the tests, evaluating the results of the tests to assign a value of "Good," "Bad," or 
"Unknown" to the results, and qualitatively qualifying the "Good" and "Bad" values according to the confidence of the 

so user in the result. (For example, the user can specify that the designated Good/Bad result is certain, somewhat certain, 
marginal, somewhat uncertain, or uncertain.) The qualifiers are then converted to numerical confidence factors which 
are used to compute evidential statistics (support and plausibility) for each conclusion provided in the model. The fault 
diagnosis process will now be described in detail with reference to Figs. 9-13. 

55 THE DIAGNOSTIC MODEL 

Referring to Fig. 9, the first step (SS 10), is to define the diagnostic model. Preferably, the diagnostic model us d 
consists of a description of the flow of information through a system to be diagnosed. Th information flow is represented 
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as a set of diagnostic tests (information sources), a s t of diagnoses (conclusions to be drawn), and a set of relationships 
between the tests and the conclusions. In addition, the model preferably includes parameters for weighting the test 
selection optimization process, grouping tests and conclusions, and specifying types of t sts to be p rformed (e.g. 
functional tests, conditional tests, or asymmetric tests). The model may be specified as a topological description of a 
5 system, the dependency model b tween failure modes and probe tests, a collection of IF. THEN... rules as in a rule- 
based xpert system, or any other formulation that describes the relationship betwe n information sources and con- 
clusions. 

Advantageously, the tool used to develop the model is the aforementioned STAMP program, which is described 
in the ARINC Research publication, STAMP User's Guide , Tech. Note 213 (March, 1987), by the inventors Sheppard 
10 and Simpson herein, and which is hereby incorporated herein by reference. STAMP can be used to develop any model 
that meets the requirement described in the previous paragraph. The models generated by STAMP permit diagnoses 
to be performed in a dynamic, context sensitive environment. Diagnostic strategies are computed based on known 
information; and can be tailored and adapted to changing conditions, such as failing test equipment, or change of 
emphasis from minimum time to fault isolate to minimum required skill level to fault isolate. 

15 

GENERATING THE DIAGNOSTIC SEQUENCE 

Once a model has been established (STAMP creates a computer readable file which is loaded into the processor 
constituting controller and test signal evaluating sections 52 and 54), the user may, optionally, specify a collection of 

20 symptoms and initial conditions (such as weighting criteria and groupings). 

In order to provide an efficient and dynamic approach to diagnosis, two elements are essential: 1) a method of 
choosing information sources that result in a conclusion being reached utilizing a minimum amount of some required 
resource; and 2) a method for inferring information from actions taken by the user in order to adjust the diagnostic 
system's knowledge of the current state of the problem. The manner in which information sources are selected so as 

25 to optimize the diagnostic process in accordance with the present invention will be discussed next. The following dis- 
cussion assumes that no grouping has been specified, no test outcomes (symptoms) have initially been determined, 
and a cost weight has been selected for information source selection optimization (i.e., the information source selection 
is biased in favor of a less expensive source (test) over a more expensive source). 

In the preferred embodiment a complete diagnostic sequence is generated in two test selection phases (step SS 

30 20). The first phase (steps 214-226 of Fig. 10) consists of selecting tests based upon the amount of information they 
provide (and subject to any other selected weighting or grouping criteria). When a diagnostic procedure is first started, 
all of the tests in the model are available for consideration and all of the possible diagnostic conclusions in the model 
are still under consideration. If symptom information (or initial conditions) are then loaded by the user, they are used 
to prune the problem space using the inference meta-rules described below. At this point, some tests are still unknown 

35 and some conclusions have no evidence denying that they are possible. The tests that provide the most discriminating 
information about the candidate conclusion set regardless of the outcome of the individual tests are selected, as de- 
scribed in more detail hereinbelow. Referring to Fig. 9, when the selected test is performed (step SS30) and the results 
of a test are evaluated to produce test data (step SS40), the test data are processed by an inference engine, to be 
described hereinafter (step SS60), and another test is chosen in the same fashion as described above. Referring to 
Fig. 10, this process continues until there are no more tests to choose (i.e. all tests have either been evaluated or their 
outcomes have been inferred) (step SS210). 

Referring still to Fig. 10, the specific process of choosing a test is based on Shannon's Theory of Information. For 
each test that is available, the amount of information that will be learned if the test passes (i.e., is good) is computed 
(step SS214). The amount of information learned from a good outcome is taken to be the number of tests that the test 

45 depends on plus the test itself. This test passed information value is then multiplied by the test's corresponding "good" 
"base confidence", a measure of the maximum confidence one can have in a good outcome, to produce a first product 
value (step SS216). 

Then the amount of information that will be learned if the test fails (i.e. is bad) is computed (step SS218). The 
amount of information learned for a bad outcome is taken to be the number of tests that depend upon the test, plus 

so the number of tests that can be designated "not needed," plus the test itself. A test may be designated as not needed 
if the test neither depends on nor is depended on by the test under consideration unless designating that test "not 
needed" creates a new ambiguity within the model. A new ambiguity is created if two or more conclusions that used 
to have different test dependency sets would have the same test dependency set because a test which is the only 
differentiator between the conclusions is the t st about to be declared not needed. 

55 Once computed, the test failed information value is then multiplied by the test's corresponding "bad" bas confi- 

dence, a measure of the maximum confidence one can have in a bad outcome, to produce a second product value 
(step SS220). Base confidences are defined by the user for both good and bad test outcomes. The two values do not 
have to be qual. 
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Next, the minimum of th first and s cond product valu s is identified for ach test (step SS222). Normalized 
weight values are then computed and multiplied by the minimums just stored to produce third product values. As with 
the base confidence values, cost and other dir ct weight valu s afl cting test s lection are also specified by the user 
(or learned from actual diagnostic experience). Direct weighting of the tests is accepted which is inversely proportional 
s to the informational value of the test, and indirect w ighting of the tests is accept d through weighting of the related 
conclusions which is directly proportional to the value of th test being weighted. As will be appreciated by those skilled 
in the art, a direct weight is a weighting factor applied directly to the tests in the model. Examples of direct weights 
include test time, test cost, or skill level required to test. An indirect weight W| ndirect [j] applied to the conclusions in a 
model are attributed to the tests as follows: 

10 

W lndireclU] = ^(W^yX^ 

where 

75 

w ( = . a user-specified indirect weight associated with conclusion c, 
tj = a test 

dj = 0; if tj does not depend on c s 
dj = 1 ; if tj does depend on c,. 

20 

Examples of indirect weights include failure rate, criticality and safety. 

It will also be appreciated that when one or more weights are applied to information values in the test selection 
process, then an emphasis factor may also be associated with any weight which has the effect of increasing the sig- 
nificance of the weight over other weights and/or the information value. In addition, if test groupings have been specified, 

2S then three additional test selection features are available. First, time and cost penalties can be applied that affect time 
and cost weights until a test group is entered (a test within the group is selected). Second, specific sequencing of tests 
within groups can be defined which overrides normal test selection choices. Third, sequencing of entire test groups 
can be defined which also override normal test selection choices. 

The normalization process results in the weights for a given criterion summing to 1 .0. The maximum of the com- 

30 puted third product values is then determined (step SS226). The test with this maximum third product value is selected 
as the next test to evaluate. 

Du ring the first phase of test selection, an optional additional form of test selection advantageously also is provided. 
In this test selection mode, the user can specify that the selection be based on a hypothesis (discussed in more detail 
hereinafter) selected by the user. If the user selects a hypothesis, then all choice overrides and modifiers are disabled, 
35 and a test is chosen based on the following rules: 

1 . A test that has a Bad outcome and depends upon the hypothesis and a minimum number of other unknown 
conclusions. 

2. A test that has a Good outcome and does not depend upon the hypothesis, but does depend upon a maximum 
40 number of other unknown conclusions. 

The first of these rules is preferred over the second. 

At the point all of the tests have a value (either given or inferred), as determined in step SS210, the second phase 
of choosing tests begins. At this point, one or more candidate failures (conclusions) have been identified and are given 

45 as a current hypothesis, as described in more detail hereinbelow. The levels of certainty associated with these failure 
conclusions are processed in the manner described in more detail hereinbelow to determine if enough evidence has 
been gathered to declare a diagnosis. If a determination is made that more evidence is necessary, and testing should 
not terminate, then another test needs to be chosen. At this point, rather than selecting a test that maximizes the 
amount of information about the system, an attempt is made to choose a test that will either separate multiple hypoth- 

50 eses or further support a single hypothesis. The purpose of this test choice process is to identify sources of evidence 
that will increase the level of certainty that a selected candidate conclusion (i.e. the current leading hypothesis, as 
defined hereinafter) is indeed the conclusion that should be drawn. 

A test is chosen based on the quantity of evidence according to a process which can include up to four evaluation 
steps. First, if more than one current hypothesis exists (step SS230), then a t st is chos n that splits the top two 

ss hypotheses (the two hypotheses with the highest associated levels of certainty) (step SS232). This is done by examining 
the dependencies and identifying which tests depend on one hypothesis and not the other hypothesis. The test that 
furnishes the most evidence and meets this criterion is selected (step SS234). If no tests exist that can separate 
hypotheses, or only one hypothesis exists, then the next evaluation step is performed. (The second and third evaluation 
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steps attempt to find a test that provides the most support for the leading hypothesis. In both of these steps, it is 
assumed that a Bad test outcome is being sought.) 

In the second evaluation step, if only on hypothesis has been select d, then the tests are valuated to find the 
test that provid s the most support for th hypothesis (step SS236). Similarly, if there are multipl curr nt hypotheses, 
s and the first step failed to choose a test, the tests are evaluated to find the test that provides the most support for the 
I ading one of the hypotheses, i.e., the current hypothesis with the highest level of certainty associated therewith. The 
intent here is to further strengthen the current or leading hypothesis, which should bring the testing sufficiency evalu- 
ation process closer to terminating the fault isolation process. Specifically, only the tests that depend upon the current 
or leading hypothesis are considered. The test that denies the most conclusions other than the current/leading hypoth- 
w esis is selected at this point (step SS238). To deny a conclusion, the test cannot depend on that conclusion. If no tests 
exist that support the current or leading hypothesis, then the next evaluation step is performed. 

In the third evaluation step, in the event that a test has not been chosen from step 2, then tests that deny the 
current or leading hypothesis are examined (step SS240). To do this, only the tests that do not depend upon the 
hypothesis conclusion are considered. The test that supports the most conclusions other than the hypothesis is selected 
is at this point (step SS242). To support a conclusion, the test must depend on that conclusion. If no tests exist that deny 
the current/leading hypothesis, then some test must be selected, and the last evaluation step is performed. 

In the last evaluation step (SS244), in the event that no tests can be chosen that meet any of the above conditions, 
then the test with the fewest trials is selected just to select some test. If this fails to select a test, then advantageously 
the user may be offered the opportunity to choose a test. Failing user selection, an end to testing is declared even if 
so the testing sufficiency evaluation otherwise indicates additional testing is appropriate. 

INFERRING INFORMATION FROM TESTING 

Referring to Fig. 9, once a test is chosen, the next step is to evaluate the test outcome to provide the test data 
2S (Good/Bad with associated confidence factor, or Unknown) (steps SS40-50). The diagnostic model is then analyzed 
using the hybrid inference engine to determine what additional information can be inferred from the new test outcome 
(step SS60). The inference engine has two parts, one employing logical inference, and one employing statistical infer- 
ence. The first, logical, part, which is only active during the first phase of fault isolation as described above, computes 
logical inferences based on the test outcomes according to i) the STAMP model assumption that there is a single fault 
30 or a limited number of specified multiple faults to be isolated and hence a single valid diagnostic conclusion; and ii) 
eight inference meta-rules (wherein a Good/pass outcome is equivalent to "false* in logic and a Bad/fail outcome is 
equivalent to "true"): 

1 . IF a test t[j] is unknown, THEN no inference is available except t[j] is unknown. 
35 2a. IF a test tlj] is false, AND test t[j] is not a negative inference test, as defined hereinafter, THEN every test upon 

which it depends is false. 

2b. IF a test t[j] is false, AND test t[j] is not a negative inference test, THEN every conclusion upon which it depends 
is false. 

3a. IF a test tfj] is true, AND test t[j] is not a positive inference test, as defined hereinafter, THEN every test that 
40 depends on tfj] is true. 

3b. IF a test t[j] is true, AND test tfj] is not a positive inference test, THEN every conclusion upon which tfj] does 
not depend is false (due to the single conclusion assumption noted hereinabove). 

3c. IF a test t[j] is true, AND test tfj] is not a positive inference test, THEN any test t[p] that is currently unknown 
and is not a dependency of tfj] is not needed, UNLESS declaring t[p] not needed creates a new ambiguity among 
45 the conclusions (due also to the single conclusion assumption). 

4. IF after rules 1-3 have been evaluated, any t[p] is unknown and depends on all of the unknown conclusions, 
THEN t[p] is true. 

5. IF after rules 1-4 have been evaluated, any t[p] is unknown and depends only on false conclusions, THEN t{p) 
is false. 

so 6. IF the outcome of test t[j] is linked, as defined hereinafter, to an outcome of tfj], THEN the outcome of t[i] is 

determined by the linkage, AND rules 1 -5 are applied to t[i]. 

7. IF a test tfj] is true, AND tfj] is a negative inference test, AND tfj] is cross-linked, as defined hereinafter, to a 
positive inference test tfj], THEN t[i] is true. 

8. IF a test is false, AND tfj] is a positive infer nee test, AND tfj] is cross-linked to a n gative inference test tfj], 
55 THEN t[i] is false. 

For purposes of these tests, a negative inference test is defined to be a test that will result in inference only if it has a 
true value. If th value of a negative inference test is false, then no inference can be made except the value of the t st. 
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A positive inference test, on the other hand, is defined to be a t st that will result in inference only if it has a false value. 
If the value of a positive inference test is true, then no inference can be made except the value of the test. The linkage 
mentioned in rule 6 refers to the situation in which a test t[j] b ing tru triggers th value of a linked test t{i] being false, 
and a t st t[j] being false triggers the value of t[i] to be true. The cross linkages mention d in rules 7 and 8 mean that 

s two tests are declared to be asymmetric (i.e. either positive or negative inference), and as such, a false value for a 
n gative inference test triggers a false value in a linked positive inference test, and a true positive inference test triggers 
a true value in a linked negative inference test. 

It will be appreciated that other sets of inference rules, or other reasoning approaches may also be used to derive 
information from the test data for use in the diagnostic process. 

10 The second, statistical, part of the inference engine functions throughout the entire diagnostic process (and thus 

in parallel with the logical part during the first phase of test selection when the inferential part is active). Several statistics 
based on the evidence gathered are computed that describe the level of certainty for each diagnostic conclusion in 
the model that it is the correct conclusion. This computation incorporates a method of attributing evidence to conclusions 
in the model to compute measures of support and plausibility for the conclusions. As is known from Dempster-Shafer 

is theory, there are two measures of the amount of evidence accumulated for a conclusion--"belief" and "plausibility." 
They are directly related to the evidence provided by tests. Support evidence from a test affects the belief of a conclu- 
sion. Belief differs from support in that it represents the total support evidence available for a given conclusion; support 
is that portion of the evidence from one test that is directed to a given conclusion. The Support Tota |[Ci) value used below 
is analogous to belief. 

20 Denial evidence affects the plausibility of a conclusion. Just as belief is the accumulation of support, plausibility is 

the accumulation of denial. However, it has a different sense than denial. Plausibility represents the upper limit on the 

belief. The relation between belief and plausibility is: 

If the belief in a proposition A is equal to Pr(A) (the probability that A is true), the denial of proposition A is equal 

to the Pr(not A), 0<Pr(not A) and Pr(A) + Pr(not A)<1 .0, 
2* then Pr(A)<1-Pr(not A) 

or belief s plausibility 

Evidence for a proposition is represented by a credibility interval (belief, plausibility]. Each interval is a subinterval 
of the probability interval [0, 1] with the lower bound (the belief) representing the degree to which the evidence supports 
the hypothesis, and the upper bound (the plausibility) represents the degree to which the evidence fails to refute the 

30 hypothesis. Under Dempster-Shafer theory, the difference between belief and plausibility represents the residual ig- 
norance or uncertainty. Under Bayesian theory, the total probability of a proposition and its negation must be one, while 
under Dempster-Shafer theory, these probabilities do not necessarily sum to 1 . The difference is uncertainty. Thus, 
the support measure quantifies the amount of supporting evidence for a given conclusion and is given by the confidence 
in the test outcome. Plausibility, on the other hand, provides a measure of the denial of conclusions and is simply one 

35 minus the support of all conclusions not supported by the test outcomes. The advantage of a Dempster-Shafer approach 
to estimating levels of certainty over a Bayesian approach is that prior probabilities do not need to be determined. 
Rather, the support and plausibility measures are determined solely by testing, the confidences in the test outcomes, 
and the known relationships between tests and conclusions. 

The computations performed by the statistical part of the inference engine are based on a modified version of the 

to Dempster-Shafer evidential reasoning process. This process associates a confidence value with each test outcome 
in the range of 0.0 ... 1 .0. As noted hereinabove, base confidences define maximum confidence values for good and 
bad outcomes respectively. In addition, the user specifies if qualifiers are to be associated with the good and bad 
outcomes. If qualifiers are not associated, then the base confidences are selected as default confidence values, e.g., 
0.999, which advantageously can be either predetermined independently of the diagnostic model or can be specified 

4S for a particular model at the time the model is being developed. If qualifiers are associated, then the user (optionally) 
lists the appropriate qualifiers in decreasing order of confidence level. Otherwise, default qualifiers, e.g., certain, some- 
what certain, marginal, somewhat uncertain, uncertain, are used. Once the qualifiers are associated, good test and 
bad test confidence values are assigned to the qualifiers as follows: 



so 



Confidence[t k ] = f>[(N-i)/(N-1)]*Base Confidence} + 0.5 



where 



ss Confidence^] = 



the good/bad confidence value for the good/bad outcome of test t k 
the corresponding good or bad bas confidence assign d to the test 
the total number of qualifiers (N>2) 
the ordinal number of the s lect d qualifier 



Base Confidenc 
N = 
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For example, given the default qualifiers and a good base confidence of 1.0, good t st confidence values would be 
assigned as follows: 



certain 


-- 1.0 


somewhat certain 


-- 0.875 


marginal 


-- 0.75 


somewhat uncertain 


-- 0.625 


uncertain 


-- 0.5 



At the outset of the statistical computations, the total accumulated support and denial values for each conclusion, 
Support Tola |[Ci] and Denial Total [cJ, respectively, and the uncertainty signal are each initialized to 0.0; and the plau- 
sibility value for each conclusion, Piausibility[C;], and the Uncertainty value are each initialized to 1 .0. If a test outcome 
is true, then that test outcome supports conclusions upon which it depends and denies conclusions upon which it does 
, s not depend. On the other hand, if the test outcome is false, then that test outcome supports conclusions upon which 
it does not depend and denies conclusions upon which it depends. The amount of support for a given conclusion i by 
a given test t k outcome is computed as follows: 

SupportlCj] = Confidence[t k ]/ICI 

20 

where 

C = set of conclusions supported 

25 ICI= size of C. 

Denial[Cj] = Confidence^ 

The above equations apply only for computing support and denial from a single source of evidence. Evidence 
furnished by multiple information sources are combined using a modified form of Dempster's Rule of Combinations. 
30 First a measure of internal conflict K is computed: 

K = ^(Support^,^]* Support^]; ioj. 

35 Next, the number of conflicts is tallied. A conflict occurs when the outcome of a test denies all of the current hypotheses, 
which hypotheses correspond to candidate signals S c , and are determined in the manner described hereinbelow. 

To determine the current hypothesis, all of the tests must have a value (good, bad, unavailable, or unneeded). At 
this point in the diagnosis procedure, the first phase of the test selection process has been completed and the second 
phase has been commenced. Since most users are at least intuitively familiar with probability, preferably the first step 

40 in determining the current hypothesis (or hypotheses) is to compute an estimate of the probability that each conclusion 
is the answer. The evidential support and plausibility statistics for each conclusion are converted to an estimated prob- 
ability value S p [ci] (signal S P ) as follows: 

SpfCj] =J4(Support Total [ Ci ] + Plausibility^]). 

The estimated probability of the UR, defined hereinafter, is treated as a special case by assuming its estimated prob- 
ability is strictly the value of Support[U] (the support for the UR calculated as set forth hereinbelow, which corresponds 
to signal S u ). This is because Plausibility[U] for the UR is always 1 .0, which would unacceptably force the estimated 
5C probability to be at least 0.5 regardless of whether any conflict has been encountered. (Although this creates potential 
problems when values are normalized, since different types of values are being normalized, the ability of the diagnostic 
tester to determine the sufficiency of testing does not appear to be adversely affected.) The conclusion with the highest 
probability is automatically considered a hypothesis (signal S CP1 ). A determination is then made whether any other 
conclusions have probabilities close enough to the maximum to be considered a hypothesis as well. A threshold function 
55 is used to make this determination: 

If S P ,<0.1, then Threshold = 0.5 * S P1 

If S P1 s0.5, then Threshold = 0.9 * S P1 

Else Threshold = S P1 * (0.4 + S P1 ). 
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If any conclusion has a probability value that exceeds th threshold, then that conclusion is also considered a hypoth sis 
(signal S CPn ). 

In accordance with the present invention, the Dempster-Shafer method is modifi d so that direct conflict betw en 
test outcomes and hypotheses results in an ov rail increase in uncertainty in th system. This modification entails the 
s introduction of a new diagnostic conclusion, r ferred to herein as the "unanticipated result" (UR). The UR is never 
denied, and it is supported only in the event of a conflict. The current support value for the UR, i.e., signal Sj, is 
computed as follows in the absence of conflict: 

ig S^SuTTests-IJ/Tests. 

In the event of a conflict, a S u value is calculated as above, and an additional computation is made using this value to 
derive the S,j signal value used to determine the sufficiency of the testing: 

15 Su = Su + (Conflicts*K*Confidence[t k ])/Tests 

where 

20 Conflicts = the number of times a conflict is encountered 

Tests = the number of tests performed and actually evaluated so far (does not include tests for which outcomes 

have been inferred; each repetition of the same test is counted) 

The new current total support for each conclusion in the model, Support Tota |[Ci], based on the accumulated test data 
25 currently inputted, is then computed: 

Support Tota| [ Ci ] = (Support^^tCj] * Support^] + 
so Uncertainty * Support[C|] + 

Support Total [Ci] * (1-Confidence[t k ]))/(1-K). 

If Support[Cj] is less than 1 /Conclusions, where the Conclusions value is the number of conclusions in the diagnostic 
35 model, then Supportfc,] is reduced by 25% to gradually remove the conclusion from consideration. Next, a new uncer- 
tainty value is computed: 

Uncertainty = (Uncertainty * (1-Confidence[t k ]))/ (1-K). 

40 

Finally, the support values Support[Cj] are normalized as indicated above. Each total support value is normalized 
to sum to 1 .0 minus the support for the UR: 

4S Support Totel [c i ]=Support Tolal [c i ] * (1 -S u )/Z j Support Total [c j ] 

In addition to computing the support values for the conclusions in the model, the denial values must also be com- 
puted. Denial is computed as follows: 

b ° Denial TeM [e,] = Denial Total [ Ci ] + Deniafc,]. 

From the denial computation, plausibility is computed: 

55 Plausibility[Cj] = 1 - (Denial Total [C;]/Tests). 

To summarize the computational flow, as shown in Fig. 1 1 , the first step (SS600) is to initialize the values for total 
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support and denial of conclusions, plausibility and uncertainty, support of the UR; and to define the initial current hy- 
potheses. (All of the possible model conclusions advantageously are selected as the initial hypotheses.) The next step 
(SS602) is to perform at st select das described hereinabove, and valuate the outcome. Th n, (SS604), a confidence 
value is assigned to the test outcome; and the support and denial for ach conclusion are calculated based on the test 

s outcome confidence value (step SS606). The next step (SS608) determines if the test outcome conflicts with all current 
hypotheses. Then the measure of internal conflict K is obtained (SS6 10) based on the old total support values and the 
pr sent test outcome support values for all of the conclusions. The present value of the UR support is then computed 
based on the number of tests evaluated, if the present test outcome is not a conflict; and based on the number of tests 
evaluated and the number of conflicts, if the present test outcome is a conflict (SS612). Next (SS614), the new total 

w support value for each conclusion is computed based on the old total support value, the present test outcome support 
value and confidence value, the old uncertainty, and K. Then a new uncertainty value is computed based on the old 
uncertainty value, the present test outcome confidence value and K (SS616). The next step (SS618) is to normalize 
the total support value for each conclusion based on the old total support value, and the present UR support value. 
Then (SS620), a new total denial is computed for each conclusion based on the old total denial and the present test 

is outcome denial value. Next (SS622), the present plausibility is computed for each conclusion based on the present 
total .denial and the number of tests evaluated. The current hypothesis/hypotheses (signals S CP ) is/are selected 
(SS624) based on the present total support and the plausibility for each conclusion. Finally (SS626), the sufficiency of 
testing is evaluated based on values derived for the two highest levels of certainty, S P1 and S P2 , and the value of the 
UR support, S u , in the manner to be described in more detail next, to determine whether to continue or terminate further 

20 testing, and whether to declare a diagnosis. 

TERMINATING THE DIAGNOSTIC PROCESS 

The testing sufficiency evaluation approach of the present invention operates to recognize when sufficient differ- 

25 entiation between levels of certainty regarding the validity of alternative candidate diagnoses exists. In the preferred 
implementation described herein, a neural network is trained to recognize patterns in the levels of certainty indicative 
of whether a diagnosis can be made or there is so much conflict that the diagnosis process should be halted. Further, 
the preferred neural network is configured to be generic, that is, it can be used without modification or retraining for 
different diagnosis (classification) problems and different diagnosis models. 

30 Referring to Fig. 12, a neural network 100 according to the present invention preferably comprises a set 110 of at 

least first, second, third and fourth input nodes 111,112, 113 and 114, respectively. Each of the input nodes has first, 
s cond and third outputs weighted by a predetermined fixed weight. Input nodes 111 and 112 respectively receive 
certainty signals S P1 and S,^ as an input. Input node 1 1 3 preferably receives uncertainty signal Su, but this input need 
not be present; and input node 114 is a bias node receiving a predetermined, e.g., unity, bias signal. 

35 |f desired, additional input nodes can be provided which receive additional certainty signals corresponding to the 

successive next highest levels of certainty associated with the model conclusions. Such additional certainty signals 
can also include signal S N referred to hereinabove corresponding to the level of certainty that some conclusion other 
than the model conclusions and the UR is the correct diagnosis. However, for the disclosed preferred embodiment, 
additional input nodes are not necessary. Since the testing sufficiency determination does not depend on the character 

40 of the conclusions constituting the leading hypothesis, only one node is needed corresponding to the highest level of 
certainty (estimated probability value in the preferred embodiment). In addition, since only a single failure is being 
isolated by any given analysis, it is only necessary to compare the highest probability value with the next highest in 
order to determine whether the highest is "high enough." 

Neural network 100 further comprises at least one set 120 of first, second, third and fourth intermediate nodes 

is 121, 122, 123 and 124 each having an output weighted by a predetermined fixed weight. Intermediate node 111 receives 
the sum of the first weighted outputs of input nodes 110 as an input, intermediate node 122 receives the sum of the 
second weighted outputs of input nodes 110 as an input, intermediate node 123 receives the sum of the third weighted 
outputs of input nodes 110 as an input, and intermediate node 124 is a bias node like node 114 receiving the prede- 
termined bias signal. At least one intermediate layer is preferred, but not necessary. Fig. 13 shows a viable two layer 

so neural network which has been implemented. 

The always active bias nodes provided in the input and intermediate layers are generally required to ensure that 
there is always some activity coming into each node of the upper layers, and to prevent singularities. 

Neural network 100 also comprises a single output node 130, responsive to the sum of said weighted outputs of 
intermediate nodes 120, for producing output signal S s indicativ of whether sufficient test data has been evaluated 

55 to declare a diagnosis. Only a single output node is needed for the preferred embodiment becaus only of simple "y s/ 
no" determination needs to b mad . 

The output weights of input and intermediate nodes 110 and 120 correspond to a model of expert data correlating 
combinations of at least certainty signal values, and preferably certainty and uncertainty signal values, with at least 
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one xpert opinion regarding the ability to d dare a diagnosis on the basis of each combination of such signal values. 
The expert data model can either be specific to a type of system being diagnosed, or as will be described in more detail 
h r inafter, can be gen ric to syst ms in g neral. 

The node output weights advantageously are obtained by training a backpropagation neural network having the 
s same architecture as said neural network using empirical c rtainty and uncertainty signal training data obtained from 
testing of real systems, and conclusions derived from at least one xpert regarding the sufficiency of the information 
represented by the combinations to declare a diagnosis for a specific system to be tested. 

Alternatively, and preferably, the backpropagation neural network is trained using normalized randomly generated 
certainty and uncertainty signal training data, and conclusions derived from at least one expert regarding the sufficiency 
10 of the information represented by said combinations to declare a diagnosis for systems in general. 

Preferably, the estimated probability values of the certainty and uncertainty signals are applied as the inputs to 
nodes 111 and 112. The training data includes first, second and third unambiguous combinations in which the data 
sufficiency determination is obvious. Advantageously, one unambiguous combination comprises certainty signal S P1 
having a probability value corresponding to 1 00% and certainty signal S P2 and uncertainty signal having zero values. 
'5 Another unambiguous combination advantageously comprises certainty signal S P1 and uncertainty signal S,j each 
havipg a probability value corresponding to 100%, and certainty signal S P2 having a zero probability value. The third 
unambiguous combination advantageously comprises both of the certainty signal S P1 , S P2 probability values having 
uniform probabilities and the uncertainty signal Sy probability value having a zero value. 

The neural network training data preferably also includes two data sets each having a predetermined number of 
20 ambiguous combinations wherein: 

both data sets have certainty signal probability values within a closed interval [0,1 ] generated at random according 
to a lognormal distribution; 

a first one of the data sets has uncertainty signal probability values generated at random according to a lognormal 
2S distribution; 

a second one of the data sets has uncertainty signal probability values set to zero; 

the values so generated are adjusted by normalizing the certainty signal probability values in each ambiguous 
combination to sum to 1 - Sy; 

all normalized certainty signal probability values and the uncertainty signal probability values less than or equal 
30 to a predetermined value based on the number of elements in the system are set to zero; 

the resultant certainty and uncertainty signal values in each of the ambiguous combinations are normalized to sum 
to the value one; and 

the larger of the resultant normalized certainty signal probability values is designated as the value input to node 
111 in each of the ambiguous combinations where the certainty signal probability values are unequal. 

35 

Preferably, the sum of the weighted outputs inputted to each of the intermediate nodes 120 and to output node 
130 is processed in the respective node with a sigmoidal activation function, which preferably is also logistic, and 
preferably the function 1/(1 +e _x ), where x is the unweighted output of a given node. 

An exemplary set of training data and the survey data from which the training data was derived are shown in the 
40 following charts: 
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NEURAL NETWORK TRAINING DATA 


Case 


Node 1 


Node 2 


Node 3 


Answer 


1 


1.00 


0.00 


0.00 


1.00 


2 


0.50 


0.00 


0.50 


1.00 


3 


0.50 


0.50 


0.00 


0.00 


4 


0.47 


0.30 


0.23 


1.00 


5 


0.59 


0.41 


0.00 


0.00 


6 


0.50 


0.50 


0.00 


0.00 


7 


0.39 


0.34 


0.27 


1.00 


8 


0.64 


0.36 


0.00 


1.00 


9 


0.58 


0.42 


0.00 


0.00 


10 


0.52 


0.48 


0.00 


0.00 


11 


0.32 


0.22 


0.46 


1.00 


12 


0.76 


0.24 


0.00 


1.00 
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(continued) 



NEURAL NETWORK TRAINING DATA 


Case 


Node 1 


Node 2 


Node 3 


Answer 


13 


0.51 


0.49 


0.00 


0.00 


14 


0.51 


0.49 


0.00 


0.00 


15 


0.56 


0.44 


0.00 


0.00 


16 


067 


0.33 


0.00 


1.00 


17 


0.66 


0.34 


0.00 


1.00 


18 


0.58 


0.42 


0.00 


0.00 


19 


0.72 


0.28 


0.00 


1.00 


20 


0.55 


0.45 


0.00 


0.00 


21 


0.70 


0.30 


0.00 


1.00 


22 


0.52 


0.48 


0.00 


0.00 


23 


0.67 


0.33 


0.00 


1.00 


24 


0.27 


0.25 


0.48 


1.00 


25 


0.50 


0.50 


0.00 


0.00 


26 


0.52 


0.48 


0.00 


0.00 


27 


0.62 


0.38 


0.00 


1.00 


28 


0.14 


0.11 


0.76 


1.00 


29 


0.43 


0.27 


0.30 


1.00 


30 


0.68 


0.32 


0.00 


1.00 


31 


0.50 


0.25 


0.25 


1.00 


32 


0.53 


0.47 


0.00 


0.00 


33 


0.51 


0.49 


0.00 


0.00 


34 


0.54 


0.46 


0.00 


0.00 


35 


0.63 


0.37 


0.00 


1.00 


36 


0.38 


0.32 


0.30 


1.00 
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ANSWER 

Clc2c2c4c5c6c7c8c9 clO 

.47 .30 .23 

.22 .36 .16 .25 

.40 .19 .40 

.39 .34 .27 

.17 .41 .23 .19 



.52 
.17 



. 16 

.18 .14 



.23 .25 
.22 .21 
.11 .13 
. 15 



.17 .20 .20 .24 .19 

.23 .54 .22 

.13 .18 .20 .22 .14 

.16 .35 .18 .14 



50 
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cl C2 C3 c4 



. 17 
. 15 
.52 



.15 
. 16 



.22 
.25 
.48 



£5 C6 
.21 

.11 



.39 .24 
.07 .06 
.50 .19 
.29 



c8 
. 20 



02 
.23 



clO UE 
.40 
.14 



.17 .19 

. 16 

.16 



. 35 
.24 



.25 
.27 



. 39 

.21 .14 



Also shown in the training data are the answers selected according to the answers provided by the polled experts 
regarding the ability to make a confident diagnoses based on the probability value combinations. Since it was desired 

35 that neural network 100 be conservative in making a termination determination, if any expert indicated that a combi- 
nation warranted termination, that answer was selected as the answer to be used in training the network. It will be 
appreciated that the survey included combinations containing ten probability values plus a UR probability value. The 
probability values were generated as indicated above, with all values less than or equal to 0.1 being set to 0.0. The 
surveyed experts were engineers who were familiar with and had used STAMP. The experts were instructed to assess 

40 the survey combinations on the basis of the following assumptions: 

1 . A system that has failed is being tested based on a diagnostic model. 

2. The c n numbers represent probabilities derived from the testing that a unit(c n ) within the system is the failure. 

3. The failure events are independent and a testability analysis has shown that each of the conclusions listed is 
45 reachable and unique in a perfect information environment. 

4. The UR value represents the probability that something has occurred in the system that the model currently 
cannot account for. 

The training data set forth hereinabove has been applied to a backpropagation neural network having random 
so initial output weight values and trained using a conventional backpropagation training program wherein each combi- 
nation of certainty and uncertainty signal values was repeatedly applied to the corresponding input nodes of the neural 
network. An error value was computed following each application corresponding to how much the actual output of node 
1 30 differed from the expected output for the combination as defined by the selected expert answer in the training data. 
Specifically, the error was computed as the squared error: 
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E = the measure of squared error 

n p = the number of patterns in the training set 

n„ = the number of output nodes 

p = the current input pattern 

i = the current output/target node 

T/p) = the expected output of node i given pattern p 

0|(p' = the actual output of node i given pattern p 

From the value E , a correction term is computed which is then propagated backward through the network, adjusting 
the node weights as it goes. The correction term is computed from the partial derivative of the error with respect to the 
weights. The weight adjustment occurs as follows: 



where 

Wji< k )(t) = the weight from node j to node i at time t for layer k 

r| = the learning rate 

Oj<p*- 1 > = the output of node j for combination p at layer k-1 

k = the current layer 

p = the current pattern 

n p = the number of patterns 

5j(P k ) = the correction term for node i at layer k resulting from pattern p 

a = momentum coefficient 

AWjj< k )(t) = change in the weight at time t. 

The correction term, 6j<P k >, is computed as follows: 
Case 1 : (the output layer) 

8 i (p - k) = (T i (p) -O i <p ' k) )O i (p ' k> (10 i (pk) ) 

Case 2: (all hidden layers) 



so Although the use of a backpropagation neural network is preferred, other types of neural networks can be used. 

For example, counterpropagation as developed by Robert Hecht-Nielson provides essentially the same functionality 
with a different architecture. Hopfield networks, Adaline networks, and other supervised learning networks could also 
be used. 

As the training data is repeatedly presented to the neural network, the network weights are modified gradually, 
55 following the error surface defin d by the network weight space down the gradient to a local minimum. When th 
n twork settles, it contains a solution to the mapping problem. 
The node output weights which resulted from this training are: 
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the weights of the first, second and third outputs of input node 111 are (-9.640), 5.049, and (-1 .624). resp ctively; 
the weights of the first, second and third outputs of input nod 112 are 15.641, (-8.491), and 2.436, respectively; 
the weights of the first, second and third outputs of input node 11 3 are (-6.680), 3.627, and (-1 .237), resp ctiv ly; 
the weights of the first, second and third outputs of input node 11 4 are (-0.422), 0.442, and (-0.427), respectiv ly; 
and 

the weights of the outputs of int rmediate nodes 121, 122, 123 and 124 are, r spectively, (-19.214), 11.457, (- 
2.483) and 4.081. 

This collection of weights has proven to make testing sufficiency determinations which correlate highly with the 
opinions of experts exposed to the same data. Specifically, neural network 100 has been tested with a set of 2601 
randomly generated test data representing all permutations of three numbers to two decimal places under the following 
constraints: 

1 . The numbers were all in the closed interval [0,1]. 

2. The numbers all summed to 1 .0. 

3. The first number was always greater than or equal to the second number. 

Twenty of these cases were randomly selected as a test survey and submitted to the same group of five experts 
used to generate the training data. On average, 4.25 of the experts agreed with neural network 100 on each test case. 
In one case, neural network 100 indicated termination was not appropriate, while four of the experts said it was appro- 
priate. This case was at the decision boundary of the neural network, and could have been decided either way. Thus, 
overall, the performance of neural network 100 was in agreement with the experts. 

It will be appreciated by those skilled in the art that the invention has been described with respect to exemplary 
embodiments, and that numerous modifications can be made therein without departing from the scope of the invention. 



Claims 

1. Automatic test data evaluation apparatus for use in a diagnostic tester that declares a diagnosis on the basis of 
localisation or isolation of failed system elements of a system having a defined number of functional elements, 
said apparatus comprising: 

means for selecting the next test to be performed on the system to be tested; 

means for generating the corresponding test data and supplying them to the system to be tested; 

means for sensing and collecting predetermined system operating parameters; 

evaluating means (54), responsive to at least one inputted test signal corresponding to test data St relating 
to at least one predetermined parameter of the system, for producing using a probabilistic evidential reasoning 
method: 

first (S CP1 ) and second (S CP2 ) candidate signals corresponding respectively to first and second possible 
diagnoses of the condition of the system respectively having the first and second highest levels of certainty 
of being valid, and 

first (S P1 ) and second (S P2 ) certainty signals corresponding respectively to values of said first and second 
highest levels of certainty; and 

characterized by testing sufficiency means (56), responsive to said first and second certainty signals, for 
producing an output signal (S s ) indicative of whether sufficient test data has been evaluated to declare a diagnosis, 
said testing sufficiency means comprising a neural network having an output node (1 30') and at least three input 
nodes (1 1 1 ', 1 1 2', 1 1 4') respectively receiving said first and second certainty signals and a predetermined bias signal 
as an input, said input nodes having weighted outputs, the output weights corresponding to a model of expert data 
correlating combinations of certainty signals with at least one expert opinion regarding the ability to declare a 
diagnosis on the basis of each combination of certainty signal values. 

2. The apparatus of claim 1 wherein: 

said evaluating means also produces an uncertainty signal S u corresponding to a m asur of the unc rtainty 
that the evaluated at least one test signal can be validly evaluated; and 
said neural n twork comprises: 
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a set (110) of first (111), s cond (112), third (113) and fourth (114) input nod s each having first, second 
and third outputs weighted by a predetermined fixed weight, said first (111) and second (112) input nodes 
respectively receiving said first (S P1 ) and s cond (S P2 ) certainty signals as an input, said third input nod 
(113) receiving said uncertainty signal (S u ), and said fourth input node (114) rec iving said predetermined 
bias signal (BIAS); 

a set (120) of first (121), second (122), third (123) and fourth (124) intermediate nodes each having an 
output weighted by a predetermined fixed weight, and said first intermediate node (121) receiving the sum 
of the first weighted outputs of said input nodes as an input, said second intermediate node (1 22) receiving 
the same of the second weighted outputs of said input nodes as an input, said third intermediate node 
(123) receiving the sum of the third weighted outputs of said input nodes as an input, and said fourth 
intermediate node (124) receiving said predetermined bias signal (BIAS); and 

an output node (130), responsive to the sum of said weighted outputs of said intermediate nodes, for 
producing said output signal (S 6 ), the output weights of said input and intermediate nodes corresponding 
to a model of expert data correlating combinations of certainty and uncertainty signal values with at least 
one expert opinion regarding the ability to declare a diagnosis on the basis of each combination of certainty 
and uncertainty signal values, said testing sufficiency means thereby also evaluating said uncertainty 
signal to produce said output signal. 

The apparatus of claim 2 wherein: 

said evaluating means also produces a signal S N corresponding to a measure of the level of certainty that a 
diagnosis not corresponding to said first or second candidate signals or to said uncertainty signal is a valid 
diagnosis; and 

said testing sufficiency means also evaluates said signal S N to produce said output signal. 

The apparatus of claim 2, wherein said evaluating means (54) comprises: 

means for storing a model of predetermined information flow relationships within the system under test; 
means for associating said test signals with sets of possible causes in accordance with said model, and for 
ascertaining the intersections of said sets of possible causes for combinations of test signals, said candidate 
signals being derived from said intersections; and 

means for determining for each candidate signal a measure of the level of certainty that it represents a valid 
diagnosis. 

The apparatus of claim 4, wherein said evaluating means (54) comprises means for determining the degree to 
which a conflict exists between at least said first candidate signal and said at least one test signal indicative of the 
level of certainty that an unanticipated result (UR) has been obtained, said uncertainty signal having a value cor- 
responding to said UR level of certainty. 

The apparatus of claim 1 wherein the values of said first and second highest levels of certainty are probability 
values. 

The apparatus of claim 2, wherein said uncertainty measure has a probability value. 

The apparatus of claim 5, wherein said UR determining means comprises: 

means for defining a plurality of conclusions corresponding to possible diagnoses of the system; 
means for assigning a confidence value to each successively inputted test signal; 

means for calculating the support and denial for each conclusion based on said confidence value of the cur- 
rently inputted test signal; 

means for determining if the currently inputted test signal conflicts with a then selected set of said conclusions 
corresponding to candidate signals; 

means for obtaining a measure ot internal conflict based on accumulated total support values and the current 
test signal support values for all of said conclusions; 

means for computing the present value of said uncertainty signal based on the number of test signals valu- 
ated, if the present test signal is not a conflict; and based on the number of test signals valuat d and the 
number of conflicts, if the present test signal is a conflict; 

means for computing a new total support value for ach conclusion based on the old total support value, the 
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present test signal support value and confidence value, an old uncertainty value, and said measure of int rnal 
conflict; 

means for computing a new uncertainty value bas d on the old uncertainty value, the present t st signal 
confid nee value and said measure of internal conflict; 
s means for normalizing the total support value for each conclusion based on the old total support value, and 

the present uncertainty signal value; 

means for computing a new total denial for each conclusion based on an old total denial and the present test 
signal denial value; 

means for computing the present plausibility for each conclusion based on the present total denial and the 
io number of test signals evaluated; 

means for selecting the current set of candidate signals based on the present total support and the plausibility 
for each conclusion. 

9. Apparatus according to claim 2, wherein said output weights of said input (111-114) and intermediate (121-124) 
is nodes have values obtained by training a backpropagation neural network having the same architecture as said 

.neural network using empirical certainty and uncertainty signal training data obtained from testing of real systems, 
and conclusions derived from at least one expert regarding the sufficiency of the information represented by said 
combinations to declare a diagnosis for a specific system to be tested. 

20 10. The apparatus of claim 9, wherein said output weights of said input (111-114) and intermediate (121-124) nodes 
have generic values obtained by training a backpropagation neural network having the same architecture as said 
neural network using normalized randomly generated certainty and uncertainty signal training data, and conclu- 
sions derived from at least one expert regarding the sufficiency of the information represented by said combinations 
to declare a diagnosis for systems in general. 

ss 

11. The apparatus of claim 5, wherein the values of said certainty and uncertainty signals correspond to probability 
values. 

12. The apparatus of claim 11 , wherein said training data includes first, second and third unambiguous combinations 
30 in which the data sufficiency determination is obvious, said first unambiguous combination comprising said first 

certainty signal having a value corresponding to 100% probability and said second certainty signal and said un- 
certainty signal having zero values, said second unambiguous combination comprising said first certainty signal 
and said uncertainty signal each having a value corresponding to 1 00% probability and said second certainty signal 
having a zero value, and said third unambiguous combination comprising both of said certainty signals having 
35 values corresponding to uniform probabilities and said uncertainty signal having a zero value. 

13. The apparatus of claim 12, wherein said training data includes two data sets each having a predetermined number 
of ambiguous combinations wherein: 

40 both data sets have certainty signal values within a closed interval [0,1] generated at random according to a 

lognormal distribution; 

a first one of said data sets has uncertainty signal values generated at random according to a lognormal 
distribution; 

a second one of said data sets has uncertainty signal values set to zero; 
45 the values so generated are adjusted by normalizing the certainty signal values in each ambiguous combination 

to sum to (1 - S u ), all normalized certainty signal values and the uncertainty signal values less than or equal 
to a predetermined value based on the number of elements in the system are set to zero; and then the resultant 
certainty and uncertainty signal values in each of said ambiguous combinations are normalized to sum to the 
value one; and 

50 the larger of the resultant normalized certainty signal values is designated as the first certainty signal value in 

each of said ambiguous combinations where the certainty signal values are unequal. 

14. The apparatus of claim 11, wherein: 

55 the sum of the weighted outputs inputted to each of the intermediate nodes and the output mode is processed 

in the r spective node with a sigmoidal activation function; 

the weights of the first, second and third outputs of the first input node are (-9.640), 5.049, and (-1.624), 
resp ctively; 
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the w ights of the first, second and third outputs of the second input node are 15.641, (-8.491), and 2.436, 
respectively; 

the weights of the first, second and third outputs of the third input node ar (-6.680), 3.627, and (-1.237), 
resp ctively; 

the weights of the first, second and third outputs of the fourth input node are (-0.422), 0.442, and (-0.427), 
respectively; 

the weight of the output of the first intermediate node is (-19.214); 
the weight of the output of the second intermediate node is 11 .457; 
the weight of the output of the third intermediate node is (-2.483); and 
the weight of the output of the fourth intermediate node is 4.081 . 

15. The apparatus of claim 14, wherein said activation function is logistic. 

16. The apparatus of claim 15, wherein said activation function is 1/(1 +e x ), where x is the unweighted output of a 
given node. 

17. A diagnostic tester for declaring a diagnosis on the basis of localisation and isolation of failed system elements of 
a system having a defined number of functional elements the tester comprising the apparatus according to any of 
the preceding claims, the tester further comprising input means (13,26,34,42,52) for supplying test signals to said 
evaluating means. 

18. The diagnostic tester of claim 17, wherein said input means comprises operator actuated input means (13,26) for 
entering test data relating to at least one predetermined aspect of the system, and for generating test signals 
corresponding to the test data. 

19. The diagnostic tester of claim 17, wherein said input means comprises automatic test means (34) responsive to 
a control signal for performing selected tests on the system, and for generating test signals corresponding to the 
test results. 

20. The diagnostic tester of claim 19, further comprising: 

control means (32) responsive to said output signal for producing said control signal to cause said automatic 
testing means to perform at least one further selected test on the system if said output signal indicates that insuf- 
ficient test data has been evaluated; and for producing a conclusion signal indicative of a diagnosis in accordance 
with said first candidate signal if said output signal indicates that sufficient test data has been evaluated. 

21. The diagnostic tester of claim 20, wherein said control means (32) produces said control signal to select said at 
least one further selected test in accordance with at least one prior evaluation of test signals. 

22. The diagnostic tester of claim 17 further comprising: 

a display (12,22,36); and 

means (24) responsive to said output signal for controlling said display to indicate the need for additional test 
data and for controlling said input means to accept a further at least one test signal for evaluation by said 
evaluating means if said output signal indicates that insufficient test data has been evaluated; and for control- 
ling said display responsive to said first candidate signal to indicate a diagnosis if said output signal indicates 
that sufficient test data has been evaluated. 

23. A diagnostic tester according to any of claims 17 to 22 and/or comprising apparatus according to any of claims 1 
to 1 6 the tester comprising an arrangement for testing a system according to a set of defined relationships between 
a plurality of sources for providing information about the system and a plurality of conclusions about the nature or 
condition of the system, said tester comprising: 

first means (34) for successively selecting (551) an information source from among the plurality of information 
sources for interrogation according to the relative amount of information provided by each information source 
until all of the information sources have been selected or the information which would b provid d th reby 
has b en determin d; 

second means (34) for successively further selecting an information source from among the plurality of infor- 
mation sources for interrogation according to which information source will increase the I vel of certainty that 
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a selected candidate conclusion constitutes a valid diagnosis of the system being tested. 

24. The diagnostic tester of claim 23 wherein said second selecting means comprises: 

means for assigning a confidence value to the information provided by an information source; 
m ans responsive to said confidence values and to th information provided by interrogated information sourc- 
es for computing a level of certainty for each of the conclusions that it constitutes a valid diagnosis; 
means for selecting candidate conclusions according to the relative values of the computed levels of certainty 
for each conclusion. 

25. The diagnostic tester of claim 24 further comprising: 

first means for identifying, if more than one candidate conclusion has been selected, which information sourc- 
es, if any, depend on one of the two candidate conclusions with the then highest levels of certainty and not the 
other, and, if more than one such information source is identified, identifying which of such information sources 
furnishes the most information about the support for the candidate conclusion on which it depends. 

26. The diagnostic tester of claim 25 further comprising: 

second means, responsive to said first identifying means determining that no information source satisfies 
the identification criteria of said first identifying means, and responsive to said selecting means having then selected 
only one candidate conclusion, for identifying the information source, if any, that provides the most support for the 
candidate conclusion having the then highest level of certainty, if more than one candidate conclusion has then 
been selected, or for the candidate conclusion, if only one candidate conclusion has then been selected. 

27. The diagnostic tester of claim 26 further comprising: 

third means, responsive to said second identifying means determining that no information source satisfies 
the identification criteria of said second identifying means, for identifying the information source, if any, that supports 
the most conclusions other than the candidate conclusion having the then highest level of certainty. 

Pat ntanspruche 

1. Automatische Testdatenauswertevorrichtung zur Verwendung in einem diagnostischen Tester, der auf Basis der 
Lokalisation oder Isolation von versagten Systemelementen eines Systems mit einer definierten Anzahl funktio- 
naler Elemente eine Diagnose abgibt, mit: 

einem Mittel zum Auswahlen des nachsten durchzufuhrenden Tests auf dem zu testenden System, 

einem Mittel zum Generieren der korrespondierenden Testdaten und zum Liefern derselben an das zu testende 
System, 

einem Mittel zum Wahrnehmen und Sammeln vorbestimmter Systembetriebsparameter, 

einem Auswertungsmittel (54), das sensibel ist fur mindestens ein eingegangenes Testsignal, das mit Test- 
daten S T korrespondiert, welches sich auf mindestens einen vorbestimmten Parameter des Systems bezieht, 
zum Produzieren unter Verwendung einer probabilistischen nachweisenden Begrundungsmethode (?), von 

ersten (S CP1 ) und zweiten (S CP2 ) Kandidatensignalen, die jeweils mit ersten und zweiten moglichen Diagnosen 
des Zustands des Systems korrespondieren und jeweils den hochsten bzw. zweithochsten Bestimmtheitsgrad, 
gultig zu sein, haben und 

ersten (S P1 ) und zweiten (S P2 ) Bestimmtheitssignalen, die jeweils mit den Werten des hochsten bzw. zweit- 
hochsten Bestimmtheitsgrads korrespondieren, und 

gekennzeichnet durch Mittel zum Testen der Hinlanglichkeit (56), sensibel fur die ersten und zweiten Bestimmt- 
heitssignale, zum Produzieren eines Ausgangssignals (S s ), das anzeigt, ob hinlanglich viele Testdaten ausg wer- 
tet worden sind, urn eine Diagnose abzugeben, wobei das Mittel zum Testen der Hinlanglichkeit ein n uronales 
Netzwerk aufweist, mit einem Ausgangsknoten (130') und mindestens drei Eingangsknoten (11V, 112', 114'), die 
jeweils die ersten und zweiten Bestimmtheitssignale und ein vorher festgesetztes Biassignal als Eingang empfan- 
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gen, wobei die Eingangsknoten gewichtete Ausgange haben, die mit einem Modell aus Exp rlendaten korr spon- 
dieren, wobei die Experlendaten Kombinationen von Bestimmtheitssignalen mit mindestens einer Expertenmei- 
nung in b zug auf die Fahigkeit, eine Diagnose auf Basis jeder Kombination von Bestimmtheitssignalwerten ab- 
zugegen, korrelieren. 

Die Vorrichtung nach Anspruch 1 , wobei 

das Auswertungsmittel auch ein Unbestimmtheitssignal (S u ) produziert, das mit einem MaB der Unbestimmt- 
heit korrespondiert, daB das ausgewertete mindestens eine Testsignal gultig ausgewertet werden kann, und 

das neuronale Netzwerk folgendes aufweist: 

einen Satz (110) erster (111), zweiter (112), dritter (113) und vierter (114) Eingangsknoten, von denen 
jeder erste, zweite und dritte Ausgange hat, die durch ein vomer festgesetztes, bestimmtes Gewicht ge- 
wichtet sind, wobei die ersten (1 1 1 ) und zweiten (112) Eingangsknoten jeweils die ersten (S P1 ) und zweiten 
(S^) Bestimmtheitssignale als Eingang empfangen, der dritte Eingangsknoten (113) das Unbestimmt- 
heitssignal (S u ) empfangt und der vierte Eingangsknoten (114) das vorher festgesetztes Biassignal (BIAS) 
empfangt, 

einen Satz (120) erster (121), zweiter (122), dritter (123) und vierter (124) mittlerer Knoten, von denen 
jeder einen durch ein vorher festgesetztes bestimmtes Gewicht gewichteten Ausgang hat, wobei der erste 
mittlere Knoten (121) die Summe der ersten gewichteten Ausgange der Eingangsknoten als Eingang 
empfangt, der zweite mittlere Knoten (122) dieselben der zweiten gewichteten Ausgange der Eingangs- 
knoten als Eingang empfangt, der dritte mittlere Knoten (123) die Summe der dritten gewichteten Aus- 
gange der Eingangsknoten als Eingang empfangt und der vierte mittlere Knoten (124) das vomer festge- 
setzte Biassignal (BIAS) empfangt, und 

einen Ausgangsknoten (130), der sensibel ist fur die Summe der gewichteten Ausgange der mittleren 
Knoten, zum Produzieren des Ausgangssignals (S s ), wobei die Ausgangsgewichte der Eingangs- und 
mittleren Knoten mit einem Modell von Expertendaten korrespondieren, wobei die Experlendaten Kom- 
binationen von Bestimmtheits- und Unbestimmtheitssignalwerten mit mindestens einer Expertenmeinung 
bezuglich der Fahigkeit korrelieren, eine Diagnose auf Basis jeder Kombination von Bestimmtheits- und 
Unbestimmtheitssignalwerten abzugeben, wodurch das Mittel zum Testen der Hinlanglichkeit das Unbe- 
stimmtheitssignal ebenfalls auswertet, urn das Ausgangssignal zu produzieren. 

Die Vorrichtung nach Anspruch 2, wobei 

das Auswertungsmittel auch ein Signal (S N ) produziert, das mit einem MaB des Bestimmtheitsgrads korre- 
spondiert, daB eine nicht mit den ersten Oder zweiten Kandidatensignalen Oder mit dem Unbestimmtheitssignal 
korrespondierende Diagnose eine gultige Diagnose ist, und 

das Mittel zum Testen der Hinlanglichkeit auch das Signal (S N ) auswertet, urn das Ausgangssignal zu produ- 
zieren. 

Die Vorrichtung nach Anspruch 2, wobei das Auswertungsmittel (54) folgendes aufweist: 

ein Mittel zum Speichern eines Modells von FluBbeziehungen vorher festgelegter Information innemalb des 
im Test befindlichen Systems, 

ein Mittel zum Assoziieren des Testsignals mit Satzen moglicher Ursachen in Ubereinstimmung mit dem Modell 
und zum Ermitteln der Schnittpunkte der Satze moglicher Ursachen fur Kombinationen von Testsignalen, wo- 
bei die Kandidatensignale von den Schnittpunkten hergeleitet werden, und 

ein Mittel zum Bestimm n eines MaBes des Bestimmtheitsgrads fur jedes Kandidatensignal, daB es eine 
gultige Diagnos reprasentiert. 

Die Vorrichtung nach Anspruch 4, wobei das Auswertungsmittel (54) Mittel aufweist zum Bestimmen des Grads, 
zu dem in Konflikt zwischen mindestens dem ersten Kandidatensignal und mindest ns einem T stsignal existiert, 
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das den Bestimmtheitsgrad anzeigt, daB ein nicht antizipiertes Ergebnis (UR) ertialt n worden ist, wobei das Un- 
bestimmtheitssignal einen mit dem UR-Bestimmtheitsgrad korrespondierenden Wert hat. 

6. Die Vorrichtung nach Anspruch 1 , wobei die Werte der hochsten und zw ithochsten Bestimmtheitsgrade Wahr- 
scheinlichkeitswerte sind. 

7. Die Vorrichtung nach Anspruch 2, wobei das UnbestimmtheitsmaB einen Wahrscheinlichkeitswert hat. 

8. Die Vorrichtung nach Anspruch 5, wobei das UR-Bestimmungsmittel folgendes aufweist: 

ein Mittel zum Definieren einer Vielzahl von SchluBfolgerungen, die mit moglichen Diagnosen des Systems 
korrespondieren, 

ein Mittel zum Zuweisen eines Vertrauenswerts zu jedem fortfolgend eingegangenen Testsignal, 

ein Mittel zum Berechnen der Zustimmung und Ablehnung fur jede SchluBfolgerung basierend auf dem Ver- 
trauenswert des gegenwartig eingegangenen Testsignals, 

ein Mittel zum Bestimmen, ob das gegenwartig eingegangene Testsignal einem dann ausgewahlten Satz der 
mit Kandidatensignalen korrespondierenden SchluBfolgerungen widerspricht, 

ein Mittel zum Bestimmen eines MaBes internen Widerspruchs basiert auf akkumulierten totalen Zustim- 
mungswerten und den gegenwartigen Testsignalstutzwerten fur alle SchluBfolgerungen, 

ein Mittel zum Berechnen des gegenwartigen Werts des Unbestimmtheitssignals basiert auf einer Anzahl 
ausgewerteter Testsignale, wenn das gegenwartige Testsignal nicht einen Widerspruch bildet, sowie basiert 
auf der Anzahl ausgewerteter Testsignale und der Anzahl Widerspruche, wenn das gegenwartige Testsignal 
einen Widerspruch bildet, 

ein Mittel zum Berechnen eines neuen totalen Zustimmungswerts fur jede SchluBfolgerung basiert auf dem 
alten totalen Zustimmungswert, dem gegenwartigen Testsignalstutzwert und Vertrauenswert, einem alten Un- 
bestimmtheitswert und dem MaB internen Widerspruchs, 

ein Mittel zum Berechnen eines neuen Unbestimmtheitswerts basiert auf dem alten Unbestimmtheitswert, 
dem gegenwartigen Testsignalvertrauenswert und dem MaB internen Widerspruchs, 

ein Mittel zum Normalisieren des totalen Zustimungswerts fur jede SchluBfolgerung basiert auf einer alten 
totalen Ablehnung und dem gegenwartigen Testsignalablehnungswert, 

ein Mittel zum Berechnen der gegenwartigen Plausibilitat fur jede SchluBfolgerung basiert auf der gegenwar- 
tigen totalen Ablehnung und der Anzahl ausgewerteter Testsignale und 

ein Mittel zum Auswahlen des gegenwartigen Satzes Kandidatensignale basiert auf der gegenwartigen totalen 
Zustimmung (auch vomer) und der Plausibilitat fOr jede SchluBfolgerung. 

9. Vorrichtung nach Anspruch 2, wobei die Ausgangsgewichte der Eingangs- (111 bis 114) und mittleren (121 bis 
1 24) Knoten Werte haben, die durch Training eines ruckwarts verzweigten neuronalen Netzwerks ermittelt worden 
sind, das die gleiche Architektur hat wie das neuronale Netzwerk, das empirische Bestimmtheits- und Unbestimm- 
heitssignal-Trainingsdaten, die beim Testen realer Systeme ermittelt wurden, sowie SchluBfolgerungen verwendet, 
die von mindestens einem Experten bezuglich der Hinlanglichkeit der von den Kombinationen reprasentierten 
Information hergeleitet sind, um eine Diagnose fur ein spezifisches zu testendes System abzugeben. 

10. Die Vorrichtung nach Anspruch 9, wobei die Ausgangsgewichte der Eingangs- (111 bis 114) und mittleren (121 
bis 124) Knoten generische Werte haben, di durch Training eines ruckwarts v rzweigten neuronalen Netzwerks 
bestimmt worden sind, das dieselbe Architektur hat wie das neuronale Netzwerk, das normalisiert , zufallig ge- 
neri rt Bestimmtheits- und Unbestimmheitssignal-Trainingsdaten verwendet sowie SchluBfolgerung n, die von 
mindestens einem Experten bezuglich der Hinlanglichkeit der von den Kombinationen reprasentierten Information 
hergeleit t worden sind, um eine Diagnos fur System im allgemeinen abzugeben. 
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Die Vorrichtung nach Anspruch 5, wobei die Werte der Bestimmth its- und Unbestimmtheitssignale mit Wahr- 
scheinlichkeitswerten korrespondieren. 

Di Vorrichtung nach Anspruch 1 1 , wobei die Trainingsdaten erste, zweite und dritte unzweideutige Kombinationen 
aulweisen, in denen die Datenhinlanglichkeitsfestlegung otfensichtlich ist, wobei die erste unzweideutige Kombi- 
nation das erste Bestimmtheitssignal aufweist, das einen Wert hat, der mit 100% Wahrscheinlichkeit korrespon- 
diert, sowie das zweite Bestimmtheitssignal und das Unbestimmtheitssignal aufweist, die Nullwerte haben, und 
wobei die zweite unzweideutige Kombination das erste Bestimmtheitssignal und das Unbestimmtheitssignal auf- 
weist, von denen jedes einen Wert hat, der mit 100% Wahrscheinlichkeit korrespondiert, sowie das zweite Be- 
stimmtheitssignal aufweist, das einen Nullwert hat, und wobei die dritte unzweideutige Kombination beide Be- 
stimmtheitssignale aufweist, die mit einheitlichen Wahrscheinlichkeiten korrespondieren, sowie das Unbestimmt- 
heitssignal aufweist, das einen Nullwert hat. 

Die Vorrichtung nach Anspruch 12, wobei die Trainingsdaten zwei Datensatze aufweist, von denen jeder eine 
vorbestimmte Anzahl zweideutiger Kombinationen hat, wobei: 

beide Datensatze Bestimmtheitssignalwerte innerhalb eines geschlossenen Intervalls [0,1] haben, die ent- 
sprechend einer logarithmischen Normalverteilung zufallig generiert sind, 

ein erster der Datensatze Unbestimmtheitssignalwerte hat, die entsprechend einer logarithmischen Normal- 
verteilung zufallig generiert sind, 

ein zweiter der Datensatze Unbestimmtheitssignalwerte hat, die zu Null gesetzt sind, 

die so generierten Werte durch Normalisierung der Bestimmtheitssignalwerte in jeder zweideutigen Kombi- 
nation eingestellt sind als Summe (1 - S u ) zu ergeben, wobei alle normalisierten Bestimmtheitssignalwerte 
und die Unbestimmtheitssignalwerte, welche kleiner Oder gleich einem vorbestimmten Wert sind, der auf der 
Anzahl der Elemente im System basiert, zu Null gesetzt sind, und dann die resultierenden Bestimmtheits- und 
Unbestimmtheitssignalwerte in jeder der zweideutigen Kombinationen normalisiert sind, urn als Summe den 
Wert eins zu ergeben, und 

der groBere der resultierenden normalisierten Bestimmtheitssignalwerte als der erste Bestimmtheitssignalwert 
in jeder der zweideutigen Kombinationen bezeichnet wird, in der die Bestimmtheitssignalwerte ungleich sind. 

Die Vorrichtung nach Anspruch 11, wobei 

die Summe der gewichteten, jedem der mittleren Knoten eingegebenen Ausgange und der Ausgangsmodus 
im jeweiligen Knoten mit einer sigmoidalen Aktivierungsfunktion verarbeitet werden, 

die Gewichte der ersten, zweiten und dritten Ausgange des ersten Eingangsknotens jeweils (-9,640), 5,049 
und (-1 ,624) betragen, 

die Gewichte des ersten, zweiten und dritten Ausgangs des zweiten Eingangsknotens jeweils 15,641, (-8,491) 
und 2,436 betragen, 

die Gewichte des ersten, zweiten und dritten Ausgangs des dritten Eingangsknotens jeweils (-6,680), 3,627 
und (-1,237) betragen, 

die Gewichte des ersten, zweiten und dritten Ausgangs des vierten Eingangsknotens jeweils (-0,422), 0,442 
und (-0,427) betragen, 

das Gewicht des Ausgangs des ersten mittleren Knotens (-19,214) betragt, 
das Gewicht des Ausgangs des zweiten mittleren Knotens 11,457 betragt, 
das Gewicht des Ausgangs des dritt n mittleren Knotens (-2,483) betragt und 
das Gewicht des Ausgangs des vierten mittleren Knotens 4,081 betragt. 
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15. Die Vorrichtung nach Anspruch 14, wobei die Aktivierungsfunklion logistisch ist. 

16. Die Vorrichtung nach Anspruch 15, wobei die Aktivierungsfunktion 1/(1-e. x ) lautet, mit x als ungewichtetem Aus- 
gang eines gegeb n n Knotens. 

s 

17. Ein diagnostischer Tester zum Abg ben einer Diagnose auf Basis der Lokalisation und Isolation versagter Sy- 
stemelemente eines Syst ms mit einer definierten Anzahl funktionaler Elemente, wobei der Tester die Vorrichtung 
nach jedem beliebigen der vorhergehenden Anspruche aufweist sowie Eingangsmittel (13, 26, 34, 42 52) zum 
Liefem von Testsignalen an das Auswertungsmittel. 

10 

18. Der diagnostische Tester nach Anspruch 17, wobei das Eingangsmittel operatoraktivierte Eingangsmittel (10, 26) 
zum Eingeben von Testdaten aufweist, die sich auf mindestens einen vorbestimmten Aspekt des Systems bezie- 
hen, sowie zum Generieren von Testsignalen, die mit den Testdaten korrespondieren. 

is 19. Der diagnostische Tester nach Anspruch 17, wobei das Eingangsmittel ein automatisches Testmittel (34) aufweist, 
das zum DurchfOhren ausgewahlter Tests auf dem System und zum Generieren von Testsignalen, die mit den 
testergebnissen korrespondieren, sensibel fur ein Kontrollsignal ist. 

20. Der diagnostische Tester nach Anspruch 19, zusatzlich mit: 

so einem Kontrollmittel (32), das zum Produzieren des Kontrollsignals sensibel fur das Ausgangssignal ist, urn das 

Kontrollsignal zu produzieren, urn zu erwirken, daB das automatische Testmittel mindestens einen weiteren aus- 
gewahlten Test aus dem System durchfuhrt, wenn das Ausgangssignal anzeigt, daB nicht hinlanglich viele Test- 
daten ausgewertet worden sind, sowie um ein SchluBfolgerungssignal zu produzieren, das in Gbereinstimmung 
mit dem ersten Kandidatensignal eine Diagnose anzeigt, wenn das Ausgangssignal anzeigt, daB hinlanglich viele 

2B Testdaten aufgewertet worden sind. 

21. Der diagnostische Tester nach Anspruch 20, wobei das Kontrollmittel (32) das Kontrollsignal zum Auswahlen des 
mindestens einen weiteren ausgewahlten Tests in Gbereinstimmung mit mindestens einer vorhergehenden Aus- 
wertung der Testsignale produziert. 

30 

22. Der diagnostische Tester nach Anspruch 17 zusatzlich mit: 

einer Anzeige (12, 22, 36) und 

35 einem Mittel (24), das sensibel fur das Ausgangssignal ist, um die Anzeige zu steuern, die Notwendigkeit 

zusatzlicher Testdaten anzuzeigen, und umdas Eingangsmittel zu steuern, mindestens ein weiteres Testsignal 
zur Auswertung durch das Auswertungsmittel zu akzeptieren, wenn das Ausgangssignal anzeigt, daB nicht 
hinlanglich viele Testdaten ausgewertet worden sind, und um die fur das erste Kandidatensignal sensible 
Anzeige zu steuern, eine Diagnose anzuzeigen, wenn das Ausgangssignal anzeigt, daB hinlanglich viele Test- 

*o daten ausgewertet worden sind. 

23. Ein diagnostischer Tester nach einem der Anspruche 17 bis 22 und/oder mit einer Vorrichtung nach einem der 
Anspruche 1 bis 16, wobei der Tester eine Anordnung zum Testen eines Systems entsprechend einem Satz defi- 
nierter Beziehungen zwischen einer Anzahl von Quellen zum Versorgen mit Informationen uber das System und 

is einer Anzahl von SchluBfolgerungen uber die Natur oder den Zustand des Systems aufweist, sowie: 

ein erstes Mittel (34) zum fortfolgenden Auswahlen (551) einer Informationsquelle unterder Anzahl der Infor- 
mationsquellen zur Abfrage entsprechend der relativen Menge an Informationen, die von jeder Informations- 
quelle geliefert ist, bis alle Informationsquellen ausgewahlt worden sind Oder die Information, die dadurch 
50 geliefert wurde, bestimmt worden ist, 

ein zweites Mittel (34) zum fortfolgenden weiteren Auswahlen einer Informationsquelle aus einer Anzahl von 
Informationsquellen zum Abfragen entsprechend dem Kriterium, welche Informationsquelle den Bestimmt- 
heitsgrad ansteigen laBt, daB in ausgewahlte SchluBfolgerung zu einem Kandidaten eine gultige Diagnose 
55 des getesteten Systems bildet. 

24. Der diagnostische Tester nach Anspruch 23, wobei das zweite Auswahlmittel folgendes aufweist: 
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ein Mittel zum Zuw isen eines Vertrauenswerts zur Information, die von ein r Informationsquelle geliefert ist, 

ein Mittel, das sensibel ist fur die Vertrauenswerte und fur die Information, di von abgefragten Informations- 
qu lien geli f rt ist, urn einen Bestimmth itsgrad fur jede der SchluBfolgerungen zu berechn n, daB er eine 
gultige Diagnose bildet, 

ein Mittel zum Auswahlen von SchluBfolgerungen zu einem Kandidaten entsprechend den relativen Werten 
der berechneten Bestimmtheitsgrade fur jede SchluBfolgerung. 



io 25. Der diagnostische Tester nach Anspruch 24 zusatzlich mit: 

einem ersten Mittel zum Identifizieren, ob mehrals eine Schlu3folgerung zu einem Kandidaten ausgewahlt worden 
ist, welche Informationsquellen, wenn uberhaupt eine, von einer der beiden SchluBfolgerungen zum Kandidaten 
mit den dann hochsten Bestimmtheitsgraden abhangen und nicht von der anderen, und, wenn mehr als eine solche 
Informationsquelle identifiziert ist, zum Identifizieren, welche der Informationsquellen die meiste Information ge- 

is wahrt uber die Zustimmung der SchluBfolgerung zum Kandidaten, von der sie abhangt. 

26. Der diagnostische Tester nach Anspruch 25 zusatzlich mit: 

einem zweiten Mittel, das sensibel ist fur das erste Identifizierungsmittel, welches festlegt, daB keine Informati- 
onsquelle dem Identifikationskriterium des ersten Identifizierungsmittels genugt, und sensibel ist fur das Auswahl- 
20 mittel, das dann nur eine SchluBfolgerung zum Kandidaten ausgewahlt hat, urn die Informationsquelle, wenn es 

eine gibt, zu identifizieren, die die meiste Zustimmung fur die SchluBfolgerung zum Kandidaten mit dem dann 
hochsten Bestimmtheitsgrad liefert, wenn dann mehr als eine SchluBfolgerung zum Kandidaten ausgewahlt wor- 
den ist, oder fur die eine SchluBfolgerung zum Kandidaten, wenn dann nur eine SchluBfolgerung zum Kandidaten 
ausgewahlt worden ist. 

2S 

27. Der diagnostische Tester nach Anspruch 26 zusatzlich mit: 

einem dritten Mittel, das sensibel ist fur das zweite Identifizierungsmittel, welches bestimmt, daB keine Informati- 
onsquelle dem Identifikationskriterium des zweiten Identifizierungsmittels zum Identifizieren der Informationsquel- 
le, wenn es eine gibt, genugt, die die meisten von der SchluBfolgerung zum Kandidaten verschiedenen 
30 SchluBfolgerungen unterstutzt, die dann den hochsten Bestimmtheitsgrad haben. 



1. Appareil automatique devaluation de donnees de test pour utilisation dans un appareil de test de diagnostic qui 
etablit un diagnostic sur la base d'une localisation ou d'une isolation d'6l§ments de systeme d6faillants d'un sys- 
teme ayant un nombre defini d'6lements fonctionnels, ledit appareil comprenant : 

des moyens pour s6lectionner le test suivant a effectuer sur le systeme a tester ; 

des moyens pour generer les donnees de test correspondantes et pour les fournir au systeme a tester ; 

des moyens pour d^tecter et recueillir des parametres predetermines de fonctionnement du systeme ; 

des moyens devaluation (54), sensibles a au moins un signal de test entr6 correspondant aux donnees de 

test S T concernant au moins un parametre pr6d6termin6 du systeme, pour produire en utilisant une m6thode 

de raisonnement 6videntiel probabiliste : 

des premier (S CP1 ) et deuxieme (S CP2 ) signaux candidats correspondant respectivement a des premier 
et deuxieme diagnostics possibles de I'etat du systeme ayant respectivement les premier et deuxieme 
plus hauts niveaux de certitude d'etre valables, et 

des premier (S P1 ) et deuxieme (S P2 ) signaux de certitude correspondant respectivement a des valeurs 
desdits premier et deuxieme plus hauts niveaux de certitude ; et 



caracterise par des moyens de suff isance des tests (56), sensibles auxdits premier et deuxieme signaux de 
certitude, pour produire un signal de sortie (S s ) indiquant si assez de donnees de test ont et6 evaluees pour 6tablir 
un diagnostic, lesdits moyens de suffisance des tests comprenant un r6seau neuronal ayant un noeud de sortie 
(130') et au moins trois noeuds d'entree (111', 112', 114') recevant respectivement lesdits premier et deuxieme 
signaux de certitude et un signal de polarisation predetermine en tant qu'entr£e, lesdits noeuds d'entree ayant 
des sorties pondSrees, les ponderations de sortie correspondant a un modele de donnees d'expert mettant en 
correlation des combinaisons de signaux de certitude avec au moins une opinion d' xpert concernant la capacit6 
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d'etablir un diagnostic sur la base de chaque combinaison de val urs de signaux de certitude. 

Appareil s Ion la revendication 1 , dans I quel : 

lesdits moyens d evaluation produisent aussi un signal d'incertitude Sj correspondant a une mesure de I'in- 
certitude qu'au moins un signal de test evalue p ut etre evalue de facon valable ; t 
ledit reseau neuronal comprend : 

un ensemble (110) de premier (111 ), deuxieme (112), troisieme (11 3) et quatrieme (114) noeuds d'entree, 
chacun ayant des premiere, deuxieme et troisieme sorties ponderees par une ponderation fixe predeter- 
mine^, lesdits premier (111) et deuxieme (112) noeuds d'entree recevant respectivement lesdits premier 
(S P1 ) et deuxieme (S P2 ) signaux de certitude en tant qu'entree, ledit troisieme (113) noeud d'entree rece- 
vant ledit signal d'incertitude (S u ), et ledit quatrieme noeud d'entree (114) recevant ledit signal de polari- 
sation predetermine (BIAS) ; 

un ensemble (120) de premier (121), deuxieme (122), troisieme (123) et quatrieme (124) noeuds inter- 
mediates, chacun ayant une sortie ponderee par une ponderation fixe predeterminee, et ledit premier 
noeud intermediaire (121 ) recevant la somme des premieres sorties ponderees desdits noeuds d'entree 
en tant qu'entree, ledit deuxieme noeud intermediaire (1 22) recevant celle des deuxiemes sorties ponde- 
rees desdits noeuds d'entree en tant qu'entree, ledit troisieme noeud intermediaire (123) recevant la som- 
me des troisiemes sorties ponderees desdits noeuds d'entree en tant qu'entree, et ledit quatrieme noeud 
intermediaire (124) recevant ledit signal de polarisation (BIAS) predetermine ; et 
un noeud de sortie (1 30), sensible a la somme desdites sorties ponderees desdits noeuds intermediaires, 
pour produire ledit signal de sortie (S s ), les ponderations de sortie desdits noeuds d'entree et interme- 
diaires correspondant a un modele de donnees d'expert mettant en correlation des combinaisons de va- 
lours de signaux de certitude et d'incertitude avec au moins une opinion d'expert concemant la capacite 
d'6tablir un diagnostic sur la base de chaque combinaison de valeurs de signaux de certitude et d'incer- 
titude, lesdits moyens de suffisance des tests evaluant ainsi aussi ledit signal d'incertitude pour produire 
ledit signal de sortie. 

Appareil selon la revendication 2, dans lequel : 

lesdits moyens devaluation produisent aussi un signal S N correspondant a une mesure du niveau de certitude 
qu'un diagnostic ne correspondant pas auxdits premier ou deuxieme signaux candidats ni audit signal d'in- 
certitude est un diagnostic valable ; et 

lesdits moyens de suffisance des tests evaluent aussi ledit signal S N pour produire ledit signal de sortie. 

Appareil selon la revendication 2, dans lequel lesdits moyens devaluation (54) comprennent : 

des moyens pour stocker un modele de relations predetermines de flux d'information dans le systeme soumis 
au test ; 

des moyens pour associer lesdits signaux de test a des ensembles de causes possibles conformement audit 
modele, et pour determiner les intersections desdits ensembles de causes possibles pour des combinaisons 
de signaux de test, lesdits signaux candidats etant derives a partir desdites intersections ; et 
des moyens pour determiner pour chaque signal candidat une mesure du niveau de certitude qu'il represente 
un diagnostic valable. 

Appareil selon la revendication 4, dans lequel lesdits moyens devaluation (54) comprennent des moyens pour 
determiner jusqu'a quel degre un conflit existe entre ledit au moins un premier signal candidat et ledit au moins 
un signal de test indiquant le niveau de certitude qu'un resultat non anticipe (UR) a ete obtenu, ledit signal d'in- 
certitude ayant une valeur correspondant audit niveau UR de certitude. 

Appareil selon la revendication 1, dans lequel les valeurs desdits premier et deuxieme plus hauts niveaux de 
certitude sont des valeurs de probabilite. 

Appareil selon la r vendication 2, dans lequel ladite mesure d'incertitude a une valeur de probabilite. 
Appareil selon la revendication 5, dans lequel lesdits moyens de determination d'UR comprennent : 
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des moyens pour definir une plurality de conclusions correspondant a des diagnostics possibles du systems ; 
des moyens pour affecter une valeur de confiance a chaque signal de test successivement entr6 ; 
des moy ns pour calculer la confirmation et I r jet pour chaqu conclusion basee sur ladite valeur d con- 
fiance du signal de test actuellement entre ; 
5 des moyens pour determiner si le signal de test actuellement entr6 est en conflit avec un ensemble, selectionne 

a ce moment-la, desdites conclusions corr spondant aux signaux candidats ; 

des moyens pour obtenir une mesure de conflit interne basee sur des valeurs cumulees de confirmation totale 
et les valeurs de confirmation du signal de test actual pour toutes lesdites conclusions ; 
des moyens pour calculer la presente valeur dudit signal d'incertitude basee sur le nombre de signaux de test 
'0 evalues, si le present signal de test n'est pas un conflit ; et, basee sur le nombre de signaux de test evalues 

et sur le nombre de conflits, si le present signal de test est un conflit ; 

des moyens pour calculer une nouvelle valeur de confirmation totale pour chaque conclusion basee sur I'an- 
cienne valeur de confirmation totale, la valeur de confirmation et la valeur de confiance du present signal de 
test, une ancienne valeur d'incertitude, et ladite mesure de conflit interne ; 
is des moyens pour calculer une nouvelle valeur d'incertitude basee sur I'ancienne valeur d'incertitude, la valeur 

t de confiance du present signal de test et ladite mesure de conflit interne ; 

des moyens pour normal iser la valeur de confirmation totale pour chaque conclusion en fonction de I'ancienne 
valeur de confirmation totale, et de la valeur du present signal d'incertitude ; 

des moyens pour calculer un nouveau rejet total pour chaque conclusion en fonction d'un ancien rejet total et 
20 de la valeur de rejet du present signal de test ; 

des moyens pour calculer la presente vraisemblance pour chaque conclusion basee sur le present rejet total 
et le nombre de signaux de test evalues ; 

des moyens pour selectionner I'ensemble actuel de signaux candidats en fonction de la presente confirmation 
totale et de la vraisemblance pour chaque conclusion. 

25 

9. Appareil selon la revendication 2, dans lequel lesdites ponderations de sortie desdits noeuds d'entr6e (111-114) 
et intermediates (121-124) ont des valeurs obtenues par apprentissage d'un reseau neuronal a propagation en 
retour, ayant la meme architecture que ledit r6seau neuronal, en utilisant des donnees d'apprentissage de signaux 
de certitude et d'incertitude empiriques obtenues par des tests sur des systemes reels, et des conclusions derivees 

30 d'au moins un expert concernant la suffisance des informations representees par lesdites combinaisons pour 

etablir un diagnostic pour un systeme specifique a tester. 

10. Appareil selon la revendication 9, dans lequel lesdites ponddrations de sortie desdits noeuds d'entr6e (111-114) 
et intermediates (121-124) ont des valeurs generiques obtenues par apprentissage d'un reseau neuronal a pro- 
as pagation en retour, ayant la meme architecture que ledit reseau neuronal, en utilisant des donnees d'apprentissage 

normalisees pour le signal de certitude et d'incertitude generees de maniere al6atoire, et des conclusions deriv6es 
d'au moins un expert concernant la suffisance des informations representees par lesdites combinaisons pour 
etablir un diagnostic pour des systemes en general. 

40 11. Appareil selon la revendication 5, dans lequel les valeurs desdits signaux de certitude et d'incertitude correspon- 
dent a des valeurs de probability. 

1 2. Appareil selon la revendication 1 1 , dans lequel lesdites donnees d'apprentissage incluent des premiere, deuxieme 
et troisieme combinaisons sans ambiguite dans lesquelles la determination de la suffisance des donnees est 

« evidente, ladite premiere combinaison sans ambiguite comprenant ledit premier signal de certitude ayant une 

valeur correspondant a une probability de 100 %, et ledit deuxieme signal de certitude et ledit signal d'incertitude 
ayant des valeurs de zero, ladite deuxieme combinaison sans ambiguite comprenant ledit premier signal de cer- 
titude et ledit signal d'incertitude, chacun ayant une valeur correspondant a une probability de 100 %, et ledit 
deuxieme signal de certitude ayant une valeur de zero, et ladite troisieme combinaison sans ambiguite comprenant 

so les deux signaux de certitude ayant des valeurs correspondant a des probabilites unifonmes et ledit signal d'incer- 

titude ayant une valeur de zero. 

13. Appareil selon la revendication 12, dans lequel lesdites donnees d'apprentissage incluent deux ensembles de 
donnees, chacun ayant un nombre predetermine de combinaisons ambigues ou : 

55 

les deux ensembles de donne s ont des valeurs de signaux de certitude dans un intervalle f rme [0,1] gene- 
rees au hasard selon une distribution log-normale ; 

un premier desdits ensembles de donnees a des valeurs de signaux d'incertitude gen6rees au hasard selon 
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une distribution log-normale ; 

un deuxieme desdits ensembles de donnees a des valeurs de signaux d'incertitude etablies a z6ro ; 
les valeurs ainsi gen§rees sont ajuste s en normalisant les valeurs de signaux de certitude dans chaque 
combinaison ambigue pour totaliser (1-Sj), toutes les valeurs normalisees de signaux de certitude et les 
valeurs de signaux d'incertitude inf6ri ures ou egales a un valeur predeterminee basee sur le nombre d'ele- 
ments dans le systeme sont etablies a zero ; et alors les valeurs r6sultantes des signaux de certitude t d'in- 
certitude dans chacune des combinaisons ambigues sont normalisees pour totaliser la valeur un ; et 
la plus grande des valeurs normalisees rSsultantes des signaux de certitude est designee comme la premiere 
valeur de signaux de certitude dans chacune desdites combinaisons ambigues dans lesquelles les valeurs 
de signaux de certitude sont inegales. 

14. Appareil selon la revendication 11, dans lequel : 

la somme des sorties pond6r6es entries a chacun des noeuds intermediates et au noeud de sortie est traitee 
dans le noeud respectif par une fonction d'activation sigmoide ; 

les ponderations des premiere, deuxieme et troisieme sorties du premier noeud d'entree sont (-9,640), 5,049, 
et (-1 ,624), respectivement ; 

les ponderations des premiere, deuxieme et troisieme sorties du deuxieme noeud d'entr6e sont 15,641, (- 
8,491), et 2,436, respectivement ; 

les ponderations des premiere, deuxieme et troisieme sorties du troisieme noeud d'entree sont (-6,680), 3,627, 
et (-1 ,237), respectivement ; 

les ponderations des premiere, deuxieme et troisieme sorties du quatrieme noeud d'entree sont (-0,422), 
0,442, et (-0,427), respectivement ; 

la ponderation de la sortie du premier noeud intermediate est (-19,214) ; 
la ponderation de la sortie du deuxieme noeud intermediate est 11 ,457 ; 
la ponderation de la sortie du troisieme noeud intermediate est (-2,483) ; et 
la ponderation de la sortie du quatrieme noeud intermediaire est 4,081 . 

15. Appareil selon la revendication 14, dans lequel ladite fonction d'activation est logistique. 

16. Appareil selon la revendication 15, dans lequel ladite fonction d'activation est 1/(1+e" x ), oil x est la sortie non 
pond6r6e d'un noeud donne. 

17. Appareil de test de diagnostic pour etablir un diagnostic sur la base d'une localisation et isolation de composants 
de systeme d6faillants d'un systeme ayant un nombre defini d'eiements fonctionnels, I'appareil de test comprenant 
I'appareil selon I'une quelconque des revendications pr6c6dentes, I'appareil de test comprenant en outre des 
moyens d'entree (13, 26, 34, 42, 52) pour fournir des signaux de test auxdits moyens devaluation. 

18. Appareil de test de diagnostic selon la revendication 17, dans lequel lesdits moyens d'entree comprennent des 
moyens d'entree (1 3, 26) actionnes par l'op6rateur pour entrer des donnees de test concernant au moins un aspect 
predetermine du systeme, et pour g6n6rer des signaux de test correspondent aux donnees de test. 

19. Appareil de test de diagnostic selon la revendication 17, dans lequel lesdits moyens d'entree comprennent des 
moyens de test automatiques (34) sensibles a un signal de commande pour effectuer des tests seiectionnes sur 
le systeme, et pour generer des signaux de test correspondant aux resultats de test. 

20. Appareil de test de diagnostic selon la revendication 19, comprenant en outre : 

des moyens de commande (32) sensibles audit signal de sortie pour produire ledit signal de commande pour 
amener lesdits moyens de test automatiques a effectuer au moins un autre test seiectionne sur le systeme si ledit 
signal de sortie indique que des donnees de test insuffisantes ont ete evaluees ; et pour produire un signal de 
conclusion indiquant un diagnostic conform6ment audit premier signal candidat si ledit signal de sortie indique 
que des donnees de test suffisantes ont 6te evaluees. 

21 . Appareil de test de diagnostic selon la revendication 20, dans lequel lesdits moyens de command (32) produisent 
ledit signal de commande pour s6lectionner ledit au moins un autre test seiectionne conform6ment a au moins 
une evaluation anterieur des signaux de test. 

22. Appareil de test de diagnostic selon la rev ndication 17, comprenant n outre : 
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unaffichage(12, 22, 36); et 

des moyens (24) sensibles audit signal de sortie pour commander ledit affichage pour indiquer le besoin pour 
des donnees de t st supplementaires et pour commander lesdits moyens d'entree de facon a accepter au 
moins un autre signal de test a evaluer par lesdits moyens devaluation si ledit signal d sortie indique qu 
s des donnees de test insuffisantes ont ete evaluees ; et pour commander ledit affichage sensible audit premier 

signal candidat de facon a indiquer un diagnostic si I dit signal de sortie indique que des donnees de test 
suffisantes ont ete evaluees. 

23. Appareil de test de diagnostic selon I'une quelconque des revendications 17 a 22, et/ou comprenant un appareil 
10 selon I'une quelconque des revendications 1 a 16, I'appareil de test comprenant un agencement pour tester un 

systeme selon un ensemble de relations definies entre une pluralite de sources pour fournir des informations 
concernant le systeme et une pluralite de conclusions concernant la nature ou I'etat du systeme, ledit appareil de 
test comprenant : 

is des premiers moyens (34) pour selectionner successivement (551 ) une source d'informations parmi la pluralite 

de sources d'informations pour I'interroger selon la quantite d'information relative fournie par chaque source 
d'informations jusqu'a ce que toutes les sources d'informations aient ete selectionnees ou que les informations 
qui seraient fournies par celles-ci aient ete determinees ; 

des deuxiemes moyens (34) pour selectionner encore une fois successivement une source d'informations 
20 parmi la pluralite de sources d'informations pour I'interroger selon la source d'informations qui augmentera le 

niveau de certitude qu'une conclusion candidate choisie constitue un diagnostic valable du systeme teste. 

24. Appareil de test de diagnostic selon la revendication 23, dans lequel lesdits deuxiemes moyens de selection 
comprennent : 

25 

des moyens pour affecter une valeur de confiance aux informations fournies par une source d'informations ; 
des moyens sensibles auxdites valeurs de confiance et aux informations fournies par des sources d'informa- 
tions interrogees pour calculer un niveau de certitude pour chacune des conclusions qu'elle constitue un dia- 
gnostic valable ; 

30 des moyens pour selectionner des conclusions candidates selon les valeurs relatives des niveaux de certitude 

calcules pour chaque conclusion. 

25. Appareil de test de diagnostic selon la revendication 24 comprenant en outre : 

des premiers moyens pour identifier, si plus d'une conclusion candidate a ete selectionnee, quelles sources 
35 d'informations, le cas echeant, dependent d'une des deux conclusions candidates avec alors les plus hauts niveaux 

de certitude et pas de I'autre, et, si plus d'une telle source d'informations est identifiee, pour identifier laquelle de 
ces sources d'informations fournit le plus d'information concernant la confirmation pour la conclusion candidate 
dont elle depend. 

40 26. Appareil de test de diagnostic selon la revendication 25 comprenant en outre : 

des deuxiemes moyens, sensibles auxdits premiers moyens d' identification determinant qu'aucune source 
d'informations ne satisfait aux criteres d'identification desdits premiers moyens d'identification, et sensibles auxdits 
moyens de selection ayant alors selectionne seulement une conclusion candidate, pour identifier la source d'in- 
formations, le cas echeant, qui fournit le plus de confirmation pour la conclusion candidate ayant alors le plus haut 

45 niveau de certitude, si plus d'une conclusion candidate a alors ete selectionnee, ou pour la conclusion candidate, 

si seulement une conclusion candidate a alors ete selectionnee. 

27. Appareil de test de diagnostic selon la revendication 26 comprenant en outre : 

des troisiemes moyens, sensibles auxdits deuxiemes moyens d'identification determinant qu'aucune source 
so d'informations ne satisfait aux criteres d'identification desdits deuxiemes moyens d'identification, pour identifier la 

source d'informations, le cas echeant, qui confirme le plus grand nombre de conclusions autres que la conclusion 
candidate ayant alors le plus haut niveau de certitude. 



55 



34 



EP 0 474 944 B1 



10 



\U □ □ □! 
AUU □ CZlj 

/]un □ tzjT 



-12 

-II 



-15 



FIG. I 

^AUTOMATIC SYSTEM TESTER 



T 



DISPLAY 22 



CPU 



INPUT INTERFACE 



-24 



-26 



CONTROLLER 
AND 

DIAGNOSTIC 
TEST SIGNAL 
ANALYZER 






V 










M 





















^30_ 



FIG.4 



32 TEST GENERATOR 



"■L 3 !- J FIG.5 



SYSTEM 

UNDER 

TEST 



SYSTEM CONTROLLER-, 



SENSOR/CONTROL 
UNIT 



42- 



L- -40 



DIAGNOSTIC 
TESTER 



SYSTEM yA 



-+44 

J FIG. 6 



35 



EP 0 474 944 B1 



^KEYBOARD 



FIG. 2 



)— 


SIGNAL 
GENERATOR 


/ 

C /-A 








CONTROL 
AND 

DIAGNOSTIC 
ANALYSIS CPU 


^displayJ 



r l7a 



.17c 



ROM 




PROCESSOR 




DATA 
MEMORY 







FIG. 3 



-52 



CONTROLLER 















s PcfT 





<; s Co 



S CP2 : 



^7" 



TO DISPLAY/ TEST GENERATOR, 
SYSTEM A ' 



FROM KEYBOARD/INPUT INTERFACE/ 
TEST GENERATOR. SYSTEM A 



TEST SIGNAL 
EVALUATOR 



T 

54 



(fpi 



S P2 S, 

FIG.7 



SUFFICIENT 

TESTING 

EVALUATOR 



^-56 



N 



36 



EP 0 474 944 B1 



Q START ^ 




COLLECT INFORMATION 
FROM SELECTED 
SOURCE(S) 



~SS2 



ANALYZE 

COLLECTED 

INFORMATION 



DRAW CANDIDATE 
CONCLUSION 



-SS3 



-SS4 



Sufficient 
information 
j:ollected^ 
? 



-SS5 



DECLARE 
DIAGNOSIS 



V 



SS6 



FIG. 8 



37 



EP 0 474 944 B1 



DEFINE MODeU~| —^-SSIO 



SELECT TEST — -SS20 



PERFORM 
SELECTED TEST 



^SS30 



EVALUATE TEST OUTCOME 

TO DERIVE GOOD/ BAD /UNKNOWN 

TEST DATA 



-SS40 



ASSIGN QUALIFIER 

TO TEST OUTCOME DATA 



PROCESS DATA WITH 
TWO PHASE LOGICAL 
AND STATISTICAL 
INFERENCE ENGINE 



-SS50 



^SS60 



NO 



^DETERMINE 
SUFFICIENCY OF 
TESTING/ABILITY 
TO DECLARE 
.DIAGNOSIS^ 



-SS70 



YES 



DECLARE DIAGNOSIS/ 
TERMINATE TESTING 



-SS80 

FIG. 9 



38 



EP 0 474 944 B1 



SS210 



SS216 



SS218- 



SS220 - 



SS222 - 



SS224 • 



SS226- 




COMBINE EACH INFO. VALUE 
W/CORRES. TEST GOOD 
BASE CONFIDENCE TO 
PRODUCE FIRST PRODUCT 
VALUE FOR EACH TEST. 



DETERMINE TEST FAILED 
INFO. VALUE FOR EACH 
TEST. 



COMBINE EACH INFO. VALUE 
W/CORRES. TEST BAD 
BASE CONFIDENCE TO 

PRODUCE SECOND PRODUCT 
VALUE FOR EACH TEST. 



COMPARE FIRST k SECOND 
PRODUCT VALUES. 
SELECT MINIMUM FOR 
EACH TEST. 



COMBINE MIN. PROD. VALUE & 
NORMALIZED WEIGHT VALUE 
FOR EACH TEST TO PRODUCE 
THIRD PRODUCT VALUES. 



FIG.IO 



DETERMINE MAX, THIRD PROD. 
VALUES AND SELECT 
CORRESPONDING TEST 





Tyes 




SELECT TEST 




FURNISHING MOST 




EVIDENCE. 




SELECT TEST DENYING 
MOST CONCLUSIONS OTHER 
THAN CURRENT/LEADING 
HYPOTHESIS. 




SELECT TEST SUPPORTING 
MOST CONCLUSIONS OTHER 
THAN CURRENT/LEADING 
HYPOTHESIS. 



SS244 V 



1 



SELECT TEST WTTH 
FEWEST TRIALS. 



39 



EP 0 474 944 B1 



INITIALIZE VALUES: 
TOTAL SUPPORT & DENIAL OF CONCLUSIONS 
PLAUSIBILITY 4 UNCERTAINTY 

SUPPORT OF UR 
DEFINE CURRENT HYPOTHESES 



SS600 

U 



PERFORM SELECTED TEST 
EVALUATE OUTCOME 



ASSIGN CONFIDENCE VALUE TO TEST OUTCOME 



CALCULATE SUPPORT & DENIAL FOR EACH 
CONCLUSION BASED ON TEST OUTCOME 



DETERMINE IF TEST OUTCOME CONFLICTS 
WITH ALL CURRENT HYPOTHESES 



MEASURE INTERNAL CONFLICT K 
BASED ON OLD TOTAL SUPPORT VALUES 
AND PRESENT TEST OUTCOME SUPPORT VALUES 



COMPUTE PRESENT VALUE OF UR SUPPORT 
BASED ON § OF TESTS EVALUATED IF PRESENT TEST 
OUTCOME NOT A CONFLICT 
BASED ON § OF TESTS EVALUATED AND § OF CONFLICTS 
IF PRESENT TEST OUTCOME IS A CONFLICT 



ZLx 



COMPUTE PRESENT 
PLAUSIBILITY FOR 
EACH CONCLUSION 
BASED ON PRESENT 
TOTAL DENIAL AND 
I OF TESTS 
EVALUATED 



SELECT CURRENT 
HYPOTHESIS/ 

HYPOTHESES BASED 

ON PRESENT TOTAL 
SUPPORT AND 
PLAUSIBILITY FOR 

EACH CONCLUSION 



DETERMINE SUFFICIENCY 
OF TESTING BASED 
ON VALUES OF TWO 
HIGHEST LEVELS 
OF CERTAINTY AND 
VALUE OF UR SUPPORT 



SS626 " 



COMPUTE NEW TOTAL SUPPORT VALUE FOR EACH 
CONCLUSION BASED ON OLD TOTAL SUPPORT 
VALUE, PRESENT TEST OUTCOME SUPPORT AND 
CONFIDENCE VALUE, OLD UNCERTAINTY, AND K 



COMPUTE NEW UNCERTAINTY VALUE BASED ON OLD 
UNCERTAINTY VALUE, PRESENT TEST OUTCOME 
CONFIDENCE VALUE AND K 



NORMALIZE TOTAL SUPPORT VALUE FOR EACH 
CONCLUSION BASED ON OLD TOTAL SUPPORT VALUE, 
AND PRESENT UR SUPPORT VALUE 



COMPUTE NEW TOTAL DENIAL FOR EACH CONCLUSION 
BASED ON OLD TOTAL DENIAL AND PRESENT TEST 
OUTCOME DENIAL VALUE 



FIG. 11 



40 



EP 0 474 944 B1 




EP 0 474 944 B1 




42 



